2024-09-13T23:31:05.537 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T23:31:05.553 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T23:31:05.650 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi/7904450 branch: reef description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_20.04}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7904450' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: reef ceph: conf: client: debug rgw: 20 setgroup: ceph setuser: ceph mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 wait-for-scrub: false ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 rgw: ec-data-pool: true erasure_code_profile: crush-failure-domain: osd k: 3 m: 1 name: testprofile frontend: beast s3tests: slow_backend: true thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - osd.0 - - mgr.x - client.0 - osd.1 - osd.2 - osd.3 seed: 4285 sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 sleep_before_teardown: 0 subset: 0/1 suite: rgw suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_e4e227b37858f5109138494e01e5ba1ba7b8ffe8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 targets: smithi012.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi167.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: conf: client: debug ms: 1 rgw gc obj min wait: 15 osd: debug ms: 1 debug objclass: 20 - rgw: client.0: null - radosgw-admin: null - radosgw-admin-rest: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_22:40:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T23:31:05.650 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_e4e227b37858f5109138494e01e5ba1ba7b8ffe8/qa; will attempt to use it 2024-09-13T23:31:05.651 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_e4e227b37858f5109138494e01e5ba1ba7b8ffe8/qa/tasks 2024-09-13T23:31:05.651 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T23:31:05.652 INFO:teuthology.task.internal:Checking packages... 2024-09-13T23:31:05.674 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8' 2024-09-13T23:31:05.674 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T23:31:05.674 INFO:teuthology.packaging:ref: None 2024-09-13T23:31:05.675 INFO:teuthology.packaging:tag: None 2024-09-13T23:31:05.675 INFO:teuthology.packaging:branch: reef 2024-09-13T23:31:05.675 INFO:teuthology.packaging:sha1: e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-13T23:31:05.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=reef 2024-09-13T23:31:05.943 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-845-ge4e227b3-1focal 2024-09-13T23:31:05.945 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T23:31:05.951 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T23:31:05.951 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T23:31:05.959 INFO:teuthology.task.internal:Saving configuration 2024-09-13T23:31:05.972 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T23:31:05.978 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T23:31:06.012 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi012.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi/7904450', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 23:24:41.226314', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T23:31:06.048 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi/7904450', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 23:24:41.224335', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-13T23:31:06.048 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T23:31:06.054 INFO:teuthology.task.internal:roles: ubuntu@smithi012.front.sepia.ceph.com - ['mon.a', 'osd.0'] 2024-09-13T23:31:06.054 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3'] 2024-09-13T23:31:06.054 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T23:31:06.134 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbf1d603130>, signals=[15]) 2024-09-13T23:31:06.134 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T23:31:06.140 INFO:teuthology.task.internal:Opening connections... 2024-09-13T23:31:06.141 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi012.front.sepia.ceph.com 2024-09-13T23:31:06.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T23:31:06.238 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2024-09-13T23:31:06.239 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T23:31:06.344 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T23:31:06.350 DEBUG:teuthology.orchestra.run.smithi012:> uname -m 2024-09-13T23:31:06.358 INFO:teuthology.orchestra.run.smithi012.stdout:x86_64 2024-09-13T23:31:06.358 DEBUG:teuthology.orchestra.run.smithi012:> cat /etc/os-release 2024-09-13T23:31:06.407 INFO:teuthology.orchestra.run.smithi012.stdout:NAME="Ubuntu" 2024-09-13T23:31:06.407 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T23:31:06.407 INFO:teuthology.orchestra.run.smithi012.stdout:ID=ubuntu 2024-09-13T23:31:06.407 INFO:teuthology.orchestra.run.smithi012.stdout:ID_LIKE=debian 2024-09-13T23:31:06.407 INFO:teuthology.orchestra.run.smithi012.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T23:31:06.407 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_ID="20.04" 2024-09-13T23:31:06.407 INFO:teuthology.orchestra.run.smithi012.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T23:31:06.408 INFO:teuthology.orchestra.run.smithi012.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T23:31:06.408 INFO:teuthology.orchestra.run.smithi012.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T23:31:06.408 INFO:teuthology.orchestra.run.smithi012.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T23:31:06.408 INFO:teuthology.orchestra.run.smithi012.stdout:VERSION_CODENAME=focal 2024-09-13T23:31:06.408 INFO:teuthology.orchestra.run.smithi012.stdout:UBUNTU_CODENAME=focal 2024-09-13T23:31:06.408 INFO:teuthology.lock.ops:Updating smithi012.front.sepia.ceph.com on lock server 2024-09-13T23:31:06.435 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2024-09-13T23:31:06.441 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2024-09-13T23:31:06.442 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2024-09-13T23:31:06.488 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Ubuntu" 2024-09-13T23:31:06.488 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-13T23:31:06.488 INFO:teuthology.orchestra.run.smithi167.stdout:ID=ubuntu 2024-09-13T23:31:06.488 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE=debian 2024-09-13T23:31:06.488 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-13T23:31:06.488 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="20.04" 2024-09-13T23:31:06.488 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T23:31:06.488 INFO:teuthology.orchestra.run.smithi167.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T23:31:06.489 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T23:31:06.489 INFO:teuthology.orchestra.run.smithi167.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T23:31:06.489 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_CODENAME=focal 2024-09-13T23:31:06.489 INFO:teuthology.orchestra.run.smithi167.stdout:UBUNTU_CODENAME=focal 2024-09-13T23:31:06.489 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2024-09-13T23:31:06.518 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T23:31:06.526 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T23:31:06.532 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T23:31:06.533 DEBUG:teuthology.orchestra.run.smithi012:> test '!' -e /home/ubuntu/cephtest 2024-09-13T23:31:06.535 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2024-09-13T23:31:06.541 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T23:31:06.546 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T23:31:06.547 DEBUG:teuthology.orchestra.run.smithi012:> test -z $(ls -A /var/lib/ceph) 2024-09-13T23:31:06.549 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2024-09-13T23:31:06.792 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T23:31:06.863 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T23:31:06.878 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T23:31:06.878 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T23:31:06.878 DEBUG:teuthology.orchestra.run.smithi012:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T23:31:06.879 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T23:31:06.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T23:31:06.884 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-09-13T23:31:06.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T23:31:06.885 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-09-13T23:31:06.891 INFO:teuthology.orchestra.run.smithi012.stdout:5.4.0-124-generic 2024-09-13T23:31:06.891 INFO:teuthology.task.kernel:Running kernel on smithi012: 5.4.0-124-generic 2024-09-13T23:31:06.891 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2024-09-13T23:31:06.892 INFO:teuthology.orchestra.run.smithi167.stdout:5.4.0-124-generic 2024-09-13T23:31:06.893 INFO:teuthology.task.kernel:Running kernel on smithi167: 5.4.0-124-generic 2024-09-13T23:31:06.893 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get clean 2024-09-13T23:31:06.987 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get update 2024-09-13T23:31:07.043 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-09-13T23:31:07.140 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T23:31:07.141 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T23:31:07.141 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-13T23:31:07.210 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-13T23:31:07.465 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T23:31:07.467 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-13T23:31:07.467 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-13T23:31:07.531 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-13T23:31:07.663 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-13T23:31:07.673 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-13T23:31:07.676 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-13T23:31:07.681 INFO:teuthology.orchestra.run.smithi167.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-13T23:31:07.681 INFO:teuthology.orchestra.run.smithi167.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-13T23:31:07.706 INFO:teuthology.orchestra.run.smithi167.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-13T23:31:07.706 INFO:teuthology.orchestra.run.smithi167.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-13T23:31:07.709 INFO:teuthology.orchestra.run.smithi167.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T23:31:07.709 INFO:teuthology.orchestra.run.smithi167.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-13T23:31:07.716 INFO:teuthology.orchestra.run.smithi167.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-13T23:31:07.722 INFO:teuthology.orchestra.run.smithi167.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-13T23:31:07.724 INFO:teuthology.orchestra.run.smithi167.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-13T23:31:07.724 INFO:teuthology.orchestra.run.smithi167.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-13T23:31:07.724 INFO:teuthology.orchestra.run.smithi167.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-13T23:31:07.730 INFO:teuthology.orchestra.run.smithi167.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-13T23:31:07.730 INFO:teuthology.orchestra.run.smithi167.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-13T23:31:07.742 INFO:teuthology.orchestra.run.smithi167.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-13T23:31:07.751 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-13T23:31:07.782 INFO:teuthology.orchestra.run.smithi167.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-13T23:31:07.786 INFO:teuthology.orchestra.run.smithi167.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-13T23:31:07.786 INFO:teuthology.orchestra.run.smithi167.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-13T23:31:07.808 INFO:teuthology.orchestra.run.smithi167.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-13T23:31:07.809 INFO:teuthology.orchestra.run.smithi167.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-13T23:31:07.812 INFO:teuthology.orchestra.run.smithi167.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T23:31:07.812 INFO:teuthology.orchestra.run.smithi167.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-13T23:31:07.821 INFO:teuthology.orchestra.run.smithi167.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-13T23:31:07.826 INFO:teuthology.orchestra.run.smithi167.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-13T23:31:07.828 INFO:teuthology.orchestra.run.smithi167.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-13T23:31:07.829 INFO:teuthology.orchestra.run.smithi167.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-13T23:31:07.829 INFO:teuthology.orchestra.run.smithi167.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-13T23:31:07.836 INFO:teuthology.orchestra.run.smithi167.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-13T23:31:07.836 INFO:teuthology.orchestra.run.smithi167.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-13T23:31:07.836 INFO:teuthology.orchestra.run.smithi167.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-13T23:31:07.836 INFO:teuthology.orchestra.run.smithi167.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-13T23:31:07.852 INFO:teuthology.orchestra.run.smithi167.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-13T23:31:07.852 INFO:teuthology.orchestra.run.smithi167.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-13T23:31:07.853 INFO:teuthology.orchestra.run.smithi167.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-13T23:31:07.853 INFO:teuthology.orchestra.run.smithi167.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-13T23:31:07.928 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,195 kB] 2024-09-13T23:31:08.073 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,030 kB] 2024-09-13T23:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,563 kB] 2024-09-13T23:31:08.434 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [812 kB] 2024-09-13T23:31:08.448 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [472 kB] 2024-09-13T23:31:08.456 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-13T23:31:08.456 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-13T23:31:08.457 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,117 kB] 2024-09-13T23:31:08.545 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [436 kB] 2024-09-13T23:31:08.552 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T23:31:08.552 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-13T23:31:08.598 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,011 kB] 2024-09-13T23:31:08.614 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-13T23:31:08.618 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.2 kB] 2024-09-13T23:31:08.618 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-13T23:31:08.619 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-13T23:31:08.619 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-13T23:31:08.634 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [550 kB] 2024-09-13T23:31:08.647 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-13T23:31:08.647 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-13T23:31:08.648 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,236 kB] 2024-09-13T23:31:08.693 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-13T23:31:08.762 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [453 kB] 2024-09-13T23:31:08.772 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-13T23:31:08.773 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [807 kB] 2024-09-13T23:31:08.788 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,230 kB] 2024-09-13T23:31:08.815 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-13T23:31:08.843 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.0 kB] 2024-09-13T23:31:08.843 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-13T23:31:08.844 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-13T23:31:08.845 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-13T23:31:08.880 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-13T23:31:08.880 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-13T23:31:08.881 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-13T23:31:08.882 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-13T23:31:08.960 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-13T23:31:08.961 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-13T23:31:08.962 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-13T23:31:12.064 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 21.8 MB in 3s (7,200 kB/s) 2024-09-13T23:31:13.096 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 21.8 MB in 4s (5,733 kB/s) 2024-09-13T23:31:13.583 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-09-13T23:31:13.609 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T23:31:13.687 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-09-13T23:31:13.905 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2024-09-13T23:31:13.905 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2024-09-13T23:31:14.110 INFO:teuthology.orchestra.run.smithi167.stdout:The following additional packages will be installed: 2024-09-13T23:31:14.110 INFO:teuthology.orchestra.run.smithi167.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T23:31:14.110 INFO:teuthology.orchestra.run.smithi167.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-13T23:31:14.110 INFO:teuthology.orchestra.run.smithi167.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-13T23:31:14.111 INFO:teuthology.orchestra.run.smithi167.stdout:Suggested packages: 2024-09-13T23:31:14.111 INFO:teuthology.orchestra.run.smithi167.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-13T23:31:14.111 INFO:teuthology.orchestra.run.smithi167.stdout:Recommended packages: 2024-09-13T23:31:14.111 INFO:teuthology.orchestra.run.smithi167.stdout: thermald 2024-09-13T23:31:14.165 INFO:teuthology.orchestra.run.smithi167.stdout:The following NEW packages will be installed: 2024-09-13T23:31:14.165 INFO:teuthology.orchestra.run.smithi167.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T23:31:14.165 INFO:teuthology.orchestra.run.smithi167.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-13T23:31:14.165 INFO:teuthology.orchestra.run.smithi167.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-13T23:31:14.166 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages will be upgraded: 2024-09-13T23:31:14.167 INFO:teuthology.orchestra.run.smithi167.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T23:31:14.350 INFO:teuthology.orchestra.run.smithi167.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T23:31:14.350 INFO:teuthology.orchestra.run.smithi167.stdout:Need to get 77.2 MB of archives. 2024-09-13T23:31:14.350 INFO:teuthology.orchestra.run.smithi167.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-13T23:31:14.350 INFO:teuthology.orchestra.run.smithi167.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-13T23:31:14.697 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-13T23:31:14.724 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T23:31:14.800 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-13T23:31:15.024 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-09-13T23:31:15.024 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-09-13T23:31:15.236 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2024-09-13T23:31:15.236 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T23:31:15.236 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-generic linux-image-5.4.0-195-generic 2024-09-13T23:31:15.236 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-5.4.0-195-generic linux-modules-extra-5.4.0-195-generic 2024-09-13T23:31:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2024-09-13T23:31:15.237 INFO:teuthology.orchestra.run.smithi012.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-13T23:31:15.237 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2024-09-13T23:31:15.237 INFO:teuthology.orchestra.run.smithi012.stdout: thermald 2024-09-13T23:31:15.301 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2024-09-13T23:31:15.301 INFO:teuthology.orchestra.run.smithi012.stdout: linux-headers-5.4.0-195 linux-headers-5.4.0-195-generic 2024-09-13T23:31:15.301 INFO:teuthology.orchestra.run.smithi012.stdout: linux-image-5.4.0-195-generic linux-modules-5.4.0-195-generic 2024-09-13T23:31:15.301 INFO:teuthology.orchestra.run.smithi012.stdout: linux-modules-extra-5.4.0-195-generic 2024-09-13T23:31:15.302 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2024-09-13T23:31:15.303 INFO:teuthology.orchestra.run.smithi012.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T23:31:15.358 INFO:teuthology.orchestra.run.smithi012.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T23:31:15.358 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 77.2 MB of archives. 2024-09-13T23:31:15.358 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-13T23:31:15.358 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-195-generic amd64 5.4.0-195.215 [15.0 MB] 2024-09-13T23:31:15.497 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-13T23:31:15.667 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-195-generic amd64 5.4.0-195.215 [10.5 MB] 2024-09-13T23:31:15.757 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-13T23:31:15.767 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-195-generic amd64 5.4.0-195.215 [39.2 MB] 2024-09-13T23:31:16.223 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-13T23:31:16.223 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-13T23:31:16.223 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-13T23:31:16.315 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-13T23:31:16.326 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-13T23:31:17.051 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 77.2 MB in 1s (76.0 MB/s) 2024-09-13T23:31:17.127 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.195.193 [1,900 B] 2024-09-13T23:31:17.127 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.195.193 [2,584 B] 2024-09-13T23:31:17.127 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195 all 5.4.0-195.215 [11.0 MB] 2024-09-13T23:31:17.244 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-13T23:31:17.405 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-195-generic amd64 5.4.0-195.215 [1,386 kB] 2024-09-13T23:31:17.465 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.195.193 [2,460 B] 2024-09-13T23:31:18.148 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 77.2 MB in 3s (23.4 MB/s) 2024-09-13T23:31:18.345 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-modules-5.4.0-195-generic. 2024-09-13T23:31:18.986 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-13T23:31:18.991 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T23:31:19.091 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:20.055 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-13T23:31:20.060 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../0-linux-modules-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T23:31:20.126 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:21.084 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-13T23:31:21.100 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T23:31:21.191 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:22.027 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-image-5.4.0-195-generic. 2024-09-13T23:31:22.042 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../1-linux-image-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T23:31:22.151 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:22.312 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-13T23:31:22.324 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T23:31:22.361 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:23.539 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-195-generic. 2024-09-13T23:31:23.554 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T23:31:23.597 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:27.194 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T23:31:27.339 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T23:31:27.654 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T23:31:27.780 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T23:31:28.125 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-13T23:31:28.139 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-13T23:31:28.170 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../3-linux-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T23:31:28.183 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T23:31:28.296 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T23:31:28.574 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T23:31:28.692 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-image-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T23:31:29.028 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-headers-5.4.0-195. 2024-09-13T23:31:29.045 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../5-linux-headers-5.4.0-195_5.4.0-195.215_all.deb ... 2024-09-13T23:31:29.086 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T23:31:33.587 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-13T23:31:33.611 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T23:31:33.673 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:34.520 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package linux-headers-5.4.0-195-generic. 2024-09-13T23:31:34.542 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../6-linux-headers-5.4.0-195-generic_5.4.0-195.215_amd64.deb ... 2024-09-13T23:31:34.585 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:36.466 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T23:31:36.619 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T23:31:36.952 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:37.168 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.195.193_amd64.deb ... 2024-09-13T23:31:37.295 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking linux-headers-generic (5.4.0.195.193) over (5.4.0.124.125) ... 2024-09-13T23:31:37.621 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-modules-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:38.199 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:39.424 INFO:teuthology.orchestra.run.smithi012.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-13T23:31:39.424 INFO:teuthology.orchestra.run.smithi012.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-13T23:31:39.476 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:39.568 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:40.794 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T23:31:40.920 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-13T23:31:41.038 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:41.197 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-13T23:31:41.215 INFO:teuthology.orchestra.run.smithi167.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-195-generic 2024-09-13T23:31:41.215 INFO:teuthology.orchestra.run.smithi167.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-195-generic 2024-09-13T23:31:41.323 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-13T23:31:41.353 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-modules-extra-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:41.450 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:41.525 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T23:31:41.525 INFO:teuthology.orchestra.run.smithi012.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-13T23:31:42.546 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-headers-5.4.0-195 (5.4.0-195.215) ... 2024-09-13T23:31:42.681 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-image-generic (5.4.0.195.193) ... 2024-09-13T23:31:42.807 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-headers-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:42.933 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-headers-generic (5.4.0.195.193) ... 2024-09-13T23:31:43.059 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up linux-generic (5.4.0.195.193) ... 2024-09-13T23:31:43.185 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for linux-image-5.4.0-195-generic (5.4.0-195.215) ... 2024-09-13T23:31:43.259 INFO:teuthology.orchestra.run.smithi167.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T23:31:43.259 INFO:teuthology.orchestra.run.smithi167.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-195-generic 2024-09-13T23:32:04.365 INFO:teuthology.orchestra.run.smithi012.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T23:32:04.365 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub' 2024-09-13T23:32:04.383 INFO:teuthology.orchestra.run.smithi012.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T23:32:04.409 INFO:teuthology.orchestra.run.smithi012.stdout:Generating grub configuration file ... 2024-09-13T23:32:04.702 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T23:32:04.743 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T23:32:05.135 INFO:teuthology.orchestra.run.smithi012.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T23:32:05.147 INFO:teuthology.orchestra.run.smithi012.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T23:32:05.276 INFO:teuthology.orchestra.run.smithi167.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T23:32:05.276 INFO:teuthology.orchestra.run.smithi167.stdout:Sourcing file `/etc/default/grub' 2024-09-13T23:32:05.288 INFO:teuthology.orchestra.run.smithi167.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T23:32:05.317 INFO:teuthology.orchestra.run.smithi167.stdout:Generating grub configuration file ... 2024-09-13T23:32:05.689 INFO:teuthology.orchestra.run.smithi167.stdout:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T23:32:05.714 INFO:teuthology.orchestra.run.smithi167.stdout:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T23:32:06.141 INFO:teuthology.orchestra.run.smithi167.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T23:32:06.152 INFO:teuthology.orchestra.run.smithi167.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T23:32:07.943 INFO:teuthology.orchestra.run.smithi012.stdout:done 2024-09-13T23:32:08.212 INFO:teuthology.orchestra.run.smithi167.stdout:done 2024-09-13T23:32:08.402 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2024-09-13T23:32:08.454 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2024-09-13T23:32:08.454 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2024-09-13T23:32:08.454 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2024-09-13T23:32:08.454 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2024-09-13T23:32:08.454 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 20 2024-09-13T23:32:08.454 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T23:32:08.454 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2024-09-13T23:32:08.454 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2024-09-13T23:32:08.455 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.4.0.195.193 2024-09-13T23:32:08.455 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T23:32:08.455 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T23:32:08.455 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2024-09-13T23:32:08.455 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2024-09-13T23:32:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2024-09-13T23:32:08.455 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2024-09-13T23:32:08.456 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-13T23:32:08.456 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T23:32:08.456 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T23:32:08.456 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T23:32:08.456 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi012.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T23:32:08.456 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get clean 2024-09-13T23:32:08.536 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-09-13T23:32:08.615 DEBUG:teuthology.orchestra.run.smithi167:> dpkg -s linux-image-generic 2024-09-13T23:32:08.657 INFO:teuthology.orchestra.run.smithi167.stdout:Package: linux-image-generic 2024-09-13T23:32:08.657 INFO:teuthology.orchestra.run.smithi167.stdout:Status: install ok installed 2024-09-13T23:32:08.657 INFO:teuthology.orchestra.run.smithi167.stdout:Priority: optional 2024-09-13T23:32:08.657 INFO:teuthology.orchestra.run.smithi167.stdout:Section: kernel 2024-09-13T23:32:08.657 INFO:teuthology.orchestra.run.smithi167.stdout:Installed-Size: 20 2024-09-13T23:32:08.658 INFO:teuthology.orchestra.run.smithi167.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T23:32:08.658 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: amd64 2024-09-13T23:32:08.658 INFO:teuthology.orchestra.run.smithi167.stdout:Source: linux-meta 2024-09-13T23:32:08.658 INFO:teuthology.orchestra.run.smithi167.stdout:Version: 5.4.0.195.193 2024-09-13T23:32:08.658 INFO:teuthology.orchestra.run.smithi167.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T23:32:08.658 INFO:teuthology.orchestra.run.smithi167.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T23:32:08.658 INFO:teuthology.orchestra.run.smithi167.stdout:Recommends: thermald 2024-09-13T23:32:08.658 INFO:teuthology.orchestra.run.smithi167.stdout:Description: Generic Linux kernel image 2024-09-13T23:32:08.658 INFO:teuthology.orchestra.run.smithi167.stdout: This package will always depend on the latest generic kernel image 2024-09-13T23:32:08.658 INFO:teuthology.orchestra.run.smithi167.stdout: available. 2024-09-13T23:32:08.658 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-195-generic 2024-09-13T23:32:08.659 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T23:32:08.659 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T23:32:08.659 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-09-13T23:32:08.659 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi167.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T23:32:08.659 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get clean 2024-09-13T23:32:08.676 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T23:32:08.738 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get update 2024-09-13T23:32:08.793 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T23:32:08.875 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T23:32:08.876 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T23:32:08.882 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T23:32:08.900 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T23:32:08.957 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T23:32:08.994 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T23:32:10.605 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-13T23:32:10.633 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T23:32:10.706 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-13T23:32:10.730 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-09-13T23:32:10.757 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T23:32:10.830 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-09-13T23:32:10.906 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-09-13T23:32:10.906 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-09-13T23:32:11.024 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2024-09-13T23:32:11.024 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2024-09-13T23:32:11.147 INFO:teuthology.orchestra.run.smithi012.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-13T23:32:11.147 INFO:teuthology.orchestra.run.smithi012.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T23:32:11.149 DEBUG:teuthology.orchestra.run.smithi012:> dpkg -s linux-image-generic 2024-09-13T23:32:11.170 INFO:teuthology.orchestra.run.smithi012.stdout:Package: linux-image-generic 2024-09-13T23:32:11.170 INFO:teuthology.orchestra.run.smithi012.stdout:Status: install ok installed 2024-09-13T23:32:11.170 INFO:teuthology.orchestra.run.smithi012.stdout:Priority: optional 2024-09-13T23:32:11.170 INFO:teuthology.orchestra.run.smithi012.stdout:Section: kernel 2024-09-13T23:32:11.170 INFO:teuthology.orchestra.run.smithi012.stdout:Installed-Size: 20 2024-09-13T23:32:11.171 INFO:teuthology.orchestra.run.smithi012.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T23:32:11.171 INFO:teuthology.orchestra.run.smithi012.stdout:Architecture: amd64 2024-09-13T23:32:11.171 INFO:teuthology.orchestra.run.smithi012.stdout:Source: linux-meta 2024-09-13T23:32:11.171 INFO:teuthology.orchestra.run.smithi012.stdout:Version: 5.4.0.195.193 2024-09-13T23:32:11.171 INFO:teuthology.orchestra.run.smithi012.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T23:32:11.171 INFO:teuthology.orchestra.run.smithi012.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T23:32:11.171 INFO:teuthology.orchestra.run.smithi012.stdout:Recommends: thermald 2024-09-13T23:32:11.171 INFO:teuthology.orchestra.run.smithi012.stdout:Description: Generic Linux kernel image 2024-09-13T23:32:11.171 INFO:teuthology.orchestra.run.smithi012.stdout: This package will always depend on the latest generic kernel image 2024-09-13T23:32:11.171 INFO:teuthology.orchestra.run.smithi012.stdout: available. 2024-09-13T23:32:11.172 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-09-13T23:32:11.220 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.VmzCPiH5P2 2024-09-13T23:32:11.220 DEBUG:teuthology.orchestra.run.smithi012:> sudo cp /boot/grub/grub.cfg /tmp/tmp.VmzCPiH5P2 2024-09-13T23:32:11.235 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.VmzCPiH5P2 2024-09-13T23:32:11.261 INFO:teuthology.orchestra.run.smithi167.stdout:linux-image-generic is already the newest version (5.4.0.195.193). 2024-09-13T23:32:11.262 INFO:teuthology.orchestra.run.smithi167.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-13T23:32:11.262 DEBUG:teuthology.orchestra.run.smithi167:> dpkg -s linux-image-generic 2024-09-13T23:32:11.275 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.VmzCPiH5P2 is 9KB 2024-09-13T23:32:11.282 INFO:teuthology.orchestra.run.smithi167.stdout:Package: linux-image-generic 2024-09-13T23:32:11.282 INFO:teuthology.orchestra.run.smithi167.stdout:Status: install ok installed 2024-09-13T23:32:11.282 INFO:teuthology.orchestra.run.smithi167.stdout:Priority: optional 2024-09-13T23:32:11.283 INFO:teuthology.orchestra.run.smithi167.stdout:Section: kernel 2024-09-13T23:32:11.283 INFO:teuthology.orchestra.run.smithi167.stdout:Installed-Size: 20 2024-09-13T23:32:11.283 INFO:teuthology.orchestra.run.smithi167.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T23:32:11.283 INFO:teuthology.orchestra.run.smithi167.stdout:Architecture: amd64 2024-09-13T23:32:11.283 INFO:teuthology.orchestra.run.smithi167.stdout:Source: linux-meta 2024-09-13T23:32:11.283 INFO:teuthology.orchestra.run.smithi167.stdout:Version: 5.4.0.195.193 2024-09-13T23:32:11.283 INFO:teuthology.orchestra.run.smithi167.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-13T23:32:11.283 INFO:teuthology.orchestra.run.smithi167.stdout:Depends: linux-image-5.4.0-195-generic, linux-modules-extra-5.4.0-195-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T23:32:11.283 INFO:teuthology.orchestra.run.smithi167.stdout:Recommends: thermald 2024-09-13T23:32:11.283 INFO:teuthology.orchestra.run.smithi167.stdout:Description: Generic Linux kernel image 2024-09-13T23:32:11.284 INFO:teuthology.orchestra.run.smithi167.stdout: This package will always depend on the latest generic kernel image 2024-09-13T23:32:11.284 INFO:teuthology.orchestra.run.smithi167.stdout: available. 2024-09-13T23:32:11.284 DEBUG:teuthology.orchestra.run.smithi167:> mktemp 2024-09-13T23:32:11.291 INFO:teuthology.orchestra.run.smithi167.stdout:/tmp/tmp.Y36wkrU0J8 2024-09-13T23:32:11.291 DEBUG:teuthology.orchestra.run.smithi167:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Y36wkrU0J8 2024-09-13T23:32:11.292 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.VmzCPiH5P2 2024-09-13T23:32:11.299 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T23:32:11.308 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0666 /tmp/tmp.Y36wkrU0J8 2024-09-13T23:32:11.347 DEBUG:teuthology.orchestra.remote:smithi167:/tmp/tmp.Y36wkrU0J8 is 9KB 2024-09-13T23:32:11.353 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:32:11.353 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T23:32:11.353 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T23:32:11.387 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-13T23:32:11.387 DEBUG:teuthology.orchestra.run.smithi012:> sudo update-grub 2024-09-13T23:32:11.400 DEBUG:teuthology.orchestra.run.smithi167:> rm -fr /tmp/tmp.Y36wkrU0J8 2024-09-13T23:32:11.406 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T23:32:11.465 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:32:11.465 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T23:32:11.465 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T23:32:11.490 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-195-generic 2024-09-13T23:32:11.490 DEBUG:teuthology.orchestra.run.smithi167:> sudo update-grub 2024-09-13T23:32:12.741 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub' 2024-09-13T23:32:12.748 INFO:teuthology.orchestra.run.smithi012.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T23:32:12.760 INFO:teuthology.orchestra.run.smithi012.stderr:Generating grub configuration file ... 2024-09-13T23:32:12.850 INFO:teuthology.orchestra.run.smithi167.stderr:Sourcing file `/etc/default/grub' 2024-09-13T23:32:12.857 INFO:teuthology.orchestra.run.smithi167.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T23:32:12.866 INFO:teuthology.orchestra.run.smithi167.stderr:Generating grub configuration file ... 2024-09-13T23:32:12.995 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T23:32:13.007 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T23:32:13.154 INFO:teuthology.orchestra.run.smithi167.stderr:Found linux image: /boot/vmlinuz-5.4.0-195-generic 2024-09-13T23:32:13.166 INFO:teuthology.orchestra.run.smithi167.stderr:Found initrd image: /boot/initrd.img-5.4.0-195-generic 2024-09-13T23:32:13.377 INFO:teuthology.orchestra.run.smithi012.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T23:32:13.389 INFO:teuthology.orchestra.run.smithi012.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T23:32:13.591 INFO:teuthology.orchestra.run.smithi167.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-13T23:32:13.603 INFO:teuthology.orchestra.run.smithi167.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-13T23:32:14.594 INFO:teuthology.orchestra.run.smithi167.stderr:done 2024-09-13T23:32:14.596 DEBUG:teuthology.orchestra.run.smithi167:> sudo shutdown -r now 2024-09-13T23:32:15.074 INFO:teuthology.orchestra.run.smithi012.stderr:done 2024-09-13T23:32:15.076 DEBUG:teuthology.orchestra.run.smithi012:> sudo shutdown -r now 2024-09-13T23:32:44.625 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-09-13T23:32:44.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-13T23:32:44.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T23:32:45.080 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T23:32:45.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-13T23:32:45.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T23:33:03.165 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-09-13T23:33:03.421 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-09-13T23:33:12.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-13T23:33:12.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T23:33:12.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-13T23:33:12.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T23:33:15.261 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.167 2024-09-13T23:33:27.271 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-13T23:33:27.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T23:33:30.813 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-09-13T23:33:42.826 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-13T23:33:42.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T23:33:45.917 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.12 2024-09-13T23:34:00.930 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-13T23:34:00.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T23:34:01.185 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-09-13T23:34:01.731 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-13T23:34:01.731 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-195-generic"... 2024-09-13T23:34:01.732 DEBUG:teuthology.orchestra.run.smithi012:> uname -r 2024-09-13T23:34:01.738 INFO:teuthology.orchestra.run.smithi012.stdout:5.4.0-195-generic 2024-09-13T23:34:01.738 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-13T23:34:01.738 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T23:34:01.738 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T23:34:02.739 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T23:34:02.739 DEBUG:teuthology.orchestra.run.smithi012:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T23:34:02.842 INFO:teuthology.orchestra.run.smithi012.stdout:ttyS1 2024-09-13T23:34:02.864 DEBUG:teuthology.parallel:result is None 2024-09-13T23:34:27.282 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T23:34:42.283 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-13T23:34:42.285 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T23:34:42.545 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-09-13T23:34:43.127 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-13T23:34:43.127 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.4.0-195-generic"... 2024-09-13T23:34:43.127 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2024-09-13T23:34:43.133 INFO:teuthology.orchestra.run.smithi167.stdout:5.4.0-195-generic 2024-09-13T23:34:43.135 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-195-generic vs 5.4.0-195-generic 2024-09-13T23:34:43.135 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T23:34:43.135 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T23:34:44.135 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-09-13T23:34:44.136 DEBUG:teuthology.orchestra.run.smithi167:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T23:34:44.236 INFO:teuthology.orchestra.run.smithi167.stdout:ttyS1 2024-09-13T23:34:44.269 DEBUG:teuthology.parallel:result is None 2024-09-13T23:34:44.269 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T23:34:44.280 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T23:34:44.281 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T23:34:44.283 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T23:34:44.290 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T23:34:44.296 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T23:34:44.303 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T23:34:44.303 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T23:34:44.305 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T23:34:44.322 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T23:34:44.328 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T23:34:44.328 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T23:34:44.360 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T23:34:44.378 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T23:34:44.381 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T23:34:44.388 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T23:34:44.393 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T23:34:44.395 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T23:34:44.402 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T23:34:44.402 DEBUG:teuthology.orchestra.run.smithi012:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T23:34:44.404 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T23:34:44.451 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T23:34:44.460 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T23:34:44.460 DEBUG:teuthology.orchestra.run.smithi012:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T23:34:44.462 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T23:34:44.468 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T23:34:44.513 DEBUG:teuthology.orchestra.run.smithi012:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T23:34:44.521 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:34:44.521 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T23:34:44.542 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T23:34:44.549 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T23:34:44.597 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:34:44.597 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T23:34:44.619 DEBUG:teuthology.orchestra.run.smithi012:> sudo service rsyslog restart 2024-09-13T23:34:44.621 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2024-09-13T23:34:44.670 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T23:34:44.677 INFO:teuthology.task.internal:Starting timer... 2024-09-13T23:34:44.677 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T23:34:44.687 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T23:34:44.697 DEBUG:teuthology.task.selinux:Excluding smithi012: OS 'ubuntu' does not support SELinux 2024-09-13T23:34:44.697 DEBUG:teuthology.task.selinux:Excluding smithi167: OS 'ubuntu' does not support SELinux 2024-09-13T23:34:44.697 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T23:34:44.697 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T23:34:44.698 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T23:34:44.698 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T23:34:44.706 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T23:34:44.717 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T23:34:44.718 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi012.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T23:40:20.466 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi012.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2024-09-13T23:40:20.468 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-13T23:40:20.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi012.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T23:40:20.579 DEBUG:teuthology.orchestra.run.smithi012:> true 2024-09-13T23:40:20.706 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi012.front.sepia.ceph.com' 2024-09-13T23:40:20.707 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-13T23:40:20.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T23:40:20.811 DEBUG:teuthology.orchestra.run.smithi167:> true 2024-09-13T23:40:20.939 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi167.front.sepia.ceph.com' 2024-09-13T23:40:20.939 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T23:40:20.950 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T23:40:20.950 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T23:40:20.950 DEBUG:teuthology.orchestra.run.smithi012:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T23:40:20.953 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T23:40:20.953 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T23:40:20.984 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:20 ntpd[13763]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T23:40:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:20 ntpd[13763]: Command line: ntpd -gq 2024-09-13T23:40:20.985 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:20 ntpd[13763]: proto: precision = 0.116 usec (-23) 2024-09-13T23:40:20.986 INFO:teuthology.orchestra.run.smithi167.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T23:40:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:20 ntpd[13763]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T23:40:20.986 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:20 ntpd[13763]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T23:40:20.986 INFO:teuthology.orchestra.run.smithi167.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T23:40:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:20 ntpd[13763]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T23:40:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:20 ntpd[13763]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T23:40:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:20 ntpd[13763]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T23:40:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:20 ntpd[13763]: Listen normally on 3 enp3s0f1 172.21.15.167:123 2024-09-13T23:40:20.987 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:20 ntpd[13763]: Listen normally on 4 lo [::1]:123 2024-09-13T23:40:20.988 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:20 ntpd[13763]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7309%5]:123 2024-09-13T23:40:20.988 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:20 ntpd[13763]: Listening on routing socket on fd #22 for interface updates 2024-09-13T23:40:20.988 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:20 ntpd[13646]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-13T23:40:20.988 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:20 ntpd[13646]: Command line: ntpd -gq 2024-09-13T23:40:20.988 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:20 ntpd[13646]: proto: precision = 0.111 usec (-23) 2024-09-13T23:40:20.989 INFO:teuthology.orchestra.run.smithi012.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T23:40:20.989 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:20 ntpd[13646]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T23:40:20.989 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:20 ntpd[13646]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T23:40:20.990 INFO:teuthology.orchestra.run.smithi012.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T23:40:20.990 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:20 ntpd[13646]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T23:40:20.990 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:20 ntpd[13646]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T23:40:20.990 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:20 ntpd[13646]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T23:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:20 ntpd[13646]: Listen normally on 3 ens1f0 172.21.15.12:123 2024-09-13T23:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:20 ntpd[13646]: Listen normally on 4 lo [::1]:123 2024-09-13T23:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:20 ntpd[13646]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:25ea%3]:123 2024-09-13T23:40:20.991 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:20 ntpd[13646]: Listening on routing socket on fd #22 for interface updates 2024-09-13T23:40:21.985 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:21 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:21.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:21 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:21.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:21 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:21.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:21 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:21.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:21 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:21.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:21 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:21.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:21 ntpd[13763]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:21.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:21 ntpd[13763]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:21 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:21 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:21.987 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:21 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:21.988 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:21 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:21.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:21 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:21.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:21 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:21.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:21 ntpd[13646]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:21.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:21 ntpd[13646]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:21.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:21 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:21.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:21 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:21.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:21 ntpd[13646]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:21.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:21 ntpd[13646]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:22.987 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:22 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:22.987 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:22 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:23.985 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:23 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:23.985 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:23 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:23.985 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:23 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:23.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:23 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:23.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:23 ntpd[13763]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:23.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:23 ntpd[13763]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:23 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:23 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:23 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:23 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:23 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:23 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:23 ntpd[13646]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:23.988 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:23 ntpd[13646]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:23.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:23 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:23.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:23 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:23.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:23 ntpd[13646]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:23.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:23 ntpd[13646]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:24.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:24 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:24.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:24 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:24.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:24 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:24.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:24 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:24.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:24 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:24.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:24 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:24.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:24 ntpd[13763]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:24.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:24 ntpd[13763]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:24.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:24 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:24.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:24 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:24.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:24 ntpd[13763]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:24.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:24 ntpd[13763]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:25.985 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:25 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:25 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:25 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:25 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:25 ntpd[13763]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:25 ntpd[13763]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:25.990 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:25.991 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:25.991 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:25.991 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:25.991 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:25.991 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:25.991 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:25.991 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:25.992 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:25 ntpd[13646]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:26.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:26 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:26.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:26 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:26.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:26 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:26.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:26 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:26.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:26 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:26.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:26 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:26.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:26 ntpd[13763]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:26.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:26 ntpd[13763]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:26.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:26 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:26.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:26 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:26.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:26 ntpd[13763]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:26.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:26 ntpd[13763]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:27.985 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:27 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:27.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:27 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:27.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:27 ntpd[13763]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:27.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:27 ntpd[13763]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:27.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:27 ntpd[13763]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:27.986 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:27 ntpd[13763]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:27.986 INFO:teuthology.orchestra.run.smithi167.stdout:13 Sep 23:40:27 ntpd[13763]: ntpd: time slew +0.003975 s 2024-09-13T23:40:27.986 INFO:teuthology.orchestra.run.smithi167.stdout:ntpd: time slew +0.003975s 2024-09-13T23:40:27.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:27 ntpd[13763]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T23:40:27.987 INFO:teuthology.orchestra.run.smithi167.stderr:13 Sep 23:40:27 ntpd[13763]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T23:40:27.989 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T23:40:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T23:40:27.990 INFO:teuthology.orchestra.run.smithi012.stdout:13 Sep 23:40:27 ntpd[13646]: ntpd: time slew +0.002743 s 2024-09-13T23:40:27.990 INFO:teuthology.orchestra.run.smithi012.stdout:ntpd: time slew +0.002743s 2024-09-13T23:40:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T23:40:27.990 INFO:teuthology.orchestra.run.smithi012.stderr:13 Sep 23:40:27 ntpd[13646]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T23:40:28.057 INFO:teuthology.orchestra.run.smithi167.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T23:40:28.057 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================== 2024-09-13T23:40:28.057 INFO:teuthology.orchestra.run.smithi167.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T23:40:28.057 INFO:teuthology.orchestra.run.smithi167.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T23:40:28.057 INFO:teuthology.orchestra.run.smithi167.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T23:40:28.057 INFO:teuthology.orchestra.run.smithi167.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T23:40:28.058 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T23:40:28.058 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2024-09-13T23:40:28.058 INFO:teuthology.orchestra.run.smithi012.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T23:40:28.058 INFO:teuthology.orchestra.run.smithi012.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T23:40:28.058 INFO:teuthology.orchestra.run.smithi012.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T23:40:28.059 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T23:40:28.059 INFO:teuthology.run_tasks:Running task install... 2024-09-13T23:40:28.068 DEBUG:teuthology.task.install:project ceph 2024-09-13T23:40:28.068 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}} 2024-09-13T23:40:28.068 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'} 2024-09-13T23:40:28.068 INFO:teuthology.task.install:Using flavor: default 2024-09-13T23:40:28.074 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T23:40:28.074 INFO:teuthology.task.install:extra packages: [] 2024-09-13T23:40:28.075 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-key list | grep Ceph 2024-09-13T23:40:28.076 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-key list | grep Ceph 2024-09-13T23:40:28.175 INFO:teuthology.orchestra.run.smithi167.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T23:40:28.202 INFO:teuthology.orchestra.run.smithi012.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-13T23:40:28.236 INFO:teuthology.orchestra.run.smithi167.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T23:40:28.236 INFO:teuthology.orchestra.run.smithi167.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T23:40:28.237 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T23:40:28.238 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-13T23:40:28.265 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T23:40:28.265 INFO:teuthology.orchestra.run.smithi012.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T23:40:28.266 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-13T23:40:28.267 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-13T23:40:28.398 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default/ 2024-09-13T23:40:28.398 INFO:teuthology.task.install.deb:Package version is 18.2.4-845-ge4e227b3-1focal 2024-09-13T23:40:28.420 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default/ 2024-09-13T23:40:28.420 INFO:teuthology.task.install.deb:Package version is 18.2.4-845-ge4e227b3-1focal 2024-09-13T23:40:28.517 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:40:28.517 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T23:40:28.533 DEBUG:teuthology.orchestra.run.smithi167:> sudo apt-get update 2024-09-13T23:40:28.536 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:40:28.537 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T23:40:28.553 DEBUG:teuthology.orchestra.run.smithi012:> sudo apt-get update 2024-09-13T23:40:28.783 INFO:teuthology.orchestra.run.smithi167.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal InRelease 2024-09-13T23:40:28.801 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T23:40:28.801 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal InRelease 2024-09-13T23:40:28.804 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T23:40:28.823 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-13T23:40:28.828 INFO:teuthology.orchestra.run.smithi167.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T23:40:28.846 INFO:teuthology.orchestra.run.smithi012.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal Release.gpg 2024-09-13T23:40:28.849 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T23:40:28.849 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main i386 Packages [3,420 B] 2024-09-13T23:40:28.859 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-13T23:40:28.868 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main i386 Packages [3,420 B] 2024-09-13T23:40:28.883 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T23:40:28.900 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 Packages [17.6 kB] 2024-09-13T23:40:28.918 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-13T23:40:28.919 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T23:40:28.942 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-13T23:40:29.001 INFO:teuthology.orchestra.run.smithi167.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T23:40:29.023 INFO:teuthology.orchestra.run.smithi012.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-13T23:40:29.478 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 28.7 kB in 1s (40.3 kB/s) 2024-09-13T23:40:29.505 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 28.7 kB in 1s (37.6 kB/s) 2024-09-13T23:40:30.667 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-09-13T23:40:30.673 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-13T23:40:30.694 DEBUG:teuthology.orchestra.run.smithi167:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-845-ge4e227b3-1focal cephadm=18.2.4-845-ge4e227b3-1focal ceph-mds=18.2.4-845-ge4e227b3-1focal ceph-mgr=18.2.4-845-ge4e227b3-1focal ceph-common=18.2.4-845-ge4e227b3-1focal ceph-fuse=18.2.4-845-ge4e227b3-1focal ceph-test=18.2.4-845-ge4e227b3-1focal ceph-volume=18.2.4-845-ge4e227b3-1focal radosgw=18.2.4-845-ge4e227b3-1focal python3-rados=18.2.4-845-ge4e227b3-1focal python3-rgw=18.2.4-845-ge4e227b3-1focal python3-cephfs=18.2.4-845-ge4e227b3-1focal python3-rbd=18.2.4-845-ge4e227b3-1focal libcephfs2=18.2.4-845-ge4e227b3-1focal libcephfs-dev=18.2.4-845-ge4e227b3-1focal librados2=18.2.4-845-ge4e227b3-1focal librbd1=18.2.4-845-ge4e227b3-1focal rbd-fuse=18.2.4-845-ge4e227b3-1focal 2024-09-13T23:40:30.699 DEBUG:teuthology.orchestra.run.smithi012:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-845-ge4e227b3-1focal cephadm=18.2.4-845-ge4e227b3-1focal ceph-mds=18.2.4-845-ge4e227b3-1focal ceph-mgr=18.2.4-845-ge4e227b3-1focal ceph-common=18.2.4-845-ge4e227b3-1focal ceph-fuse=18.2.4-845-ge4e227b3-1focal ceph-test=18.2.4-845-ge4e227b3-1focal ceph-volume=18.2.4-845-ge4e227b3-1focal radosgw=18.2.4-845-ge4e227b3-1focal python3-rados=18.2.4-845-ge4e227b3-1focal python3-rgw=18.2.4-845-ge4e227b3-1focal python3-cephfs=18.2.4-845-ge4e227b3-1focal python3-rbd=18.2.4-845-ge4e227b3-1focal libcephfs2=18.2.4-845-ge4e227b3-1focal libcephfs-dev=18.2.4-845-ge4e227b3-1focal librados2=18.2.4-845-ge4e227b3-1focal librbd1=18.2.4-845-ge4e227b3-1focal rbd-fuse=18.2.4-845-ge4e227b3-1focal 2024-09-13T23:40:30.770 INFO:teuthology.orchestra.run.smithi167.stdout:Reading package lists... 2024-09-13T23:40:30.776 INFO:teuthology.orchestra.run.smithi012.stdout:Reading package lists... 2024-09-13T23:40:30.988 INFO:teuthology.orchestra.run.smithi167.stdout:Building dependency tree... 2024-09-13T23:40:30.988 INFO:teuthology.orchestra.run.smithi167.stdout:Reading state information... 2024-09-13T23:40:31.001 INFO:teuthology.orchestra.run.smithi012.stdout:Building dependency tree... 2024-09-13T23:40:31.002 INFO:teuthology.orchestra.run.smithi012.stdout:Reading state information... 2024-09-13T23:40:31.186 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T23:40:31.188 INFO:teuthology.orchestra.run.smithi167.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T23:40:31.188 INFO:teuthology.orchestra.run.smithi167.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T23:40:31.188 INFO:teuthology.orchestra.run.smithi167.stdout:The following additional packages will be installed: 2024-09-13T23:40:31.188 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T23:40:31.188 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T23:40:31.188 INFO:teuthology.orchestra.run.smithi167.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T23:40:31.189 INFO:teuthology.orchestra.run.smithi167.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T23:40:31.189 INFO:teuthology.orchestra.run.smithi167.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T23:40:31.189 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T23:40:31.190 INFO:teuthology.orchestra.run.smithi167.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T23:40:31.190 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T23:40:31.190 INFO:teuthology.orchestra.run.smithi167.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T23:40:31.190 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2024-09-13T23:40:31.190 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-09-13T23:40:31.190 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T23:40:31.190 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T23:40:31.190 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T23:40:31.190 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T23:40:31.190 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-13T23:40:31.190 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T23:40:31.191 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T23:40:31.191 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T23:40:31.191 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T23:40:31.191 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T23:40:31.191 INFO:teuthology.orchestra.run.smithi167.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T23:40:31.191 INFO:teuthology.orchestra.run.smithi167.stdout:Suggested packages: 2024-09-13T23:40:31.191 INFO:teuthology.orchestra.run.smithi167.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T23:40:31.191 INFO:teuthology.orchestra.run.smithi167.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T23:40:31.191 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T23:40:31.192 INFO:teuthology.orchestra.run.smithi167.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T23:40:31.192 INFO:teuthology.orchestra.run.smithi167.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T23:40:31.192 INFO:teuthology.orchestra.run.smithi167.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T23:40:31.192 INFO:teuthology.orchestra.run.smithi167.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T23:40:31.192 INFO:teuthology.orchestra.run.smithi167.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T23:40:31.192 INFO:teuthology.orchestra.run.smithi167.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T23:40:31.192 INFO:teuthology.orchestra.run.smithi167.stdout:Recommended packages: 2024-09-13T23:40:31.192 INFO:teuthology.orchestra.run.smithi167.stdout: btrfs-tools 2024-09-13T23:40:31.211 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T23:40:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-13T23:40:31.212 INFO:teuthology.orchestra.run.smithi012.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T23:40:31.212 INFO:teuthology.orchestra.run.smithi012.stdout:The following additional packages will be installed: 2024-09-13T23:40:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T23:40:31.212 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-13T23:40:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-13T23:40:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-13T23:40:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-13T23:40:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-13T23:40:31.213 INFO:teuthology.orchestra.run.smithi012.stdout: python-asyncssh-doc python-natsort-doc python-pastedeploy-tpl 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-asyncssh python3-atomicwrites python3-attr python3-bcrypt 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-ceph-common python3-cherrypy3 python3-cryptography python3-decorator 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-repoze.lru python3-requests-oauthlib python3-routes python3-rsa 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-scipy python3-simplegeneric python3-simplejson 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-13T23:40:31.214 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-13T23:40:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T23:40:31.215 INFO:teuthology.orchestra.run.smithi012.stdout:Suggested packages: 2024-09-13T23:40:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-13T23:40:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-13T23:40:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-13T23:40:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-13T23:40:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-13T23:40:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-13T23:40:31.215 INFO:teuthology.orchestra.run.smithi012.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-13T23:40:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-13T23:40:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-13T23:40:31.216 INFO:teuthology.orchestra.run.smithi012.stdout:Recommended packages: 2024-09-13T23:40:31.216 INFO:teuthology.orchestra.run.smithi012.stdout: btrfs-tools 2024-09-13T23:40:31.312 INFO:teuthology.orchestra.run.smithi167.stdout:The following NEW packages will be installed: 2024-09-13T23:40:31.312 INFO:teuthology.orchestra.run.smithi167.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T23:40:31.312 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T23:40:31.312 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T23:40:31.312 INFO:teuthology.orchestra.run.smithi167.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-13T23:40:31.312 INFO:teuthology.orchestra.run.smithi167.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-13T23:40:31.312 INFO:teuthology.orchestra.run.smithi167.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-13T23:40:31.313 INFO:teuthology.orchestra.run.smithi167.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-13T23:40:31.313 INFO:teuthology.orchestra.run.smithi167.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-asyncssh-doc 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python-natsort-doc python-pastedeploy-tpl python3-asyncssh 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-09-13T23:40:31.314 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-13T23:40:31.315 INFO:teuthology.orchestra.run.smithi167.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-09-13T23:40:31.315 INFO:teuthology.orchestra.run.smithi167.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-09-13T23:40:31.315 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest python3-werkzeug qttranslations5-l10n radosgw rbd-fuse socat 2024-09-13T23:40:31.315 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet 2024-09-13T23:40:31.315 INFO:teuthology.orchestra.run.smithi167.stdout:The following packages will be upgraded: 2024-09-13T23:40:31.315 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 librbd1 2024-09-13T23:40:31.336 INFO:teuthology.orchestra.run.smithi012.stdout:The following NEW packages will be installed: 2024-09-13T23:40:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T23:40:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T23:40:31.336 INFO:teuthology.orchestra.run.smithi012.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T23:40:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-13T23:40:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-13T23:40:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-13T23:40:31.337 INFO:teuthology.orchestra.run.smithi012.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-13T23:40:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-asyncssh-doc 2024-09-13T23:40:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: python-natsort-doc python-pastedeploy-tpl python3-asyncssh 2024-09-13T23:40:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2024-09-13T23:40:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T23:40:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: python3-cherrypy3 python3-cryptography python3-decorator python3-google-auth 2024-09-13T23:40:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: python3-html5lib python3-jinja2 python3-joblib python3-jwt 2024-09-13T23:40:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-09-13T23:40:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: python3-markupsafe python3-natsort python3-oauthlib python3-openssl 2024-09-13T23:40:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2024-09-13T23:40:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2024-09-13T23:40:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pytest 2024-09-13T23:40:31.338 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2024-09-13T23:40:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2024-09-13T23:40:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-13T23:40:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2024-09-13T23:40:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2024-09-13T23:40:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: python3-webtest python3-werkzeug qttranslations5-l10n radosgw rbd-fuse socat 2024-09-13T23:40:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: xmlstarlet 2024-09-13T23:40:31.339 INFO:teuthology.orchestra.run.smithi012.stdout:The following packages will be upgraded: 2024-09-13T23:40:31.339 INFO:teuthology.orchestra.run.smithi012.stdout: librados2 librbd1 2024-09-13T23:40:31.380 INFO:teuthology.orchestra.run.smithi167.stdout:2 upgraded, 110 newly installed, 0 to remove and 295 not upgraded. 2024-09-13T23:40:31.380 INFO:teuthology.orchestra.run.smithi167.stdout:Need to get 199 MB of archives. 2024-09-13T23:40:31.380 INFO:teuthology.orchestra.run.smithi167.stdout:After this operation, 977 MB of additional disk space will be used. 2024-09-13T23:40:31.380 INFO:teuthology.orchestra.run.smithi167.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T23:40:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:2 upgraded, 110 newly installed, 0 to remove and 295 not upgraded. 2024-09-13T23:40:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:Need to get 199 MB of archives. 2024-09-13T23:40:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:After this operation, 977 MB of additional disk space will be used. 2024-09-13T23:40:31.405 INFO:teuthology.orchestra.run.smithi012.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-13T23:40:31.464 INFO:teuthology.orchestra.run.smithi167.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T23:40:31.464 INFO:teuthology.orchestra.run.smithi167.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-845-ge4e227b3-1focal [3,451 kB] 2024-09-13T23:40:31.466 INFO:teuthology.orchestra.run.smithi167.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T23:40:31.484 INFO:teuthology.orchestra.run.smithi167.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T23:40:31.490 INFO:teuthology.orchestra.run.smithi012.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 18.2.4-845-ge4e227b3-1focal [3,451 kB] 2024-09-13T23:40:31.490 INFO:teuthology.orchestra.run.smithi012.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-13T23:40:31.493 INFO:teuthology.orchestra.run.smithi012.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-13T23:40:31.511 INFO:teuthology.orchestra.run.smithi012.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-13T23:40:31.537 INFO:teuthology.orchestra.run.smithi167.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T23:40:31.539 INFO:teuthology.orchestra.run.smithi167.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T23:40:31.546 INFO:teuthology.orchestra.run.smithi167.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T23:40:31.548 INFO:teuthology.orchestra.run.smithi167.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T23:40:31.548 INFO:teuthology.orchestra.run.smithi167.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T23:40:31.549 INFO:teuthology.orchestra.run.smithi167.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T23:40:31.549 INFO:teuthology.orchestra.run.smithi167.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T23:40:31.553 INFO:teuthology.orchestra.run.smithi167.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T23:40:31.553 INFO:teuthology.orchestra.run.smithi167.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T23:40:31.555 INFO:teuthology.orchestra.run.smithi167.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T23:40:31.566 INFO:teuthology.orchestra.run.smithi012.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-13T23:40:31.568 INFO:teuthology.orchestra.run.smithi012.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-13T23:40:31.572 INFO:teuthology.orchestra.run.smithi167.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T23:40:31.572 INFO:teuthology.orchestra.run.smithi167.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T23:40:31.573 INFO:teuthology.orchestra.run.smithi167.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T23:40:31.574 INFO:teuthology.orchestra.run.smithi012.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-13T23:40:31.575 INFO:teuthology.orchestra.run.smithi167.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T23:40:31.575 INFO:teuthology.orchestra.run.smithi167.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T23:40:31.576 INFO:teuthology.orchestra.run.smithi167.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T23:40:31.576 INFO:teuthology.orchestra.run.smithi012.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-13T23:40:31.576 INFO:teuthology.orchestra.run.smithi012.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-13T23:40:31.577 INFO:teuthology.orchestra.run.smithi012.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-13T23:40:31.578 INFO:teuthology.orchestra.run.smithi167.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T23:40:31.578 INFO:teuthology.orchestra.run.smithi012.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-13T23:40:31.578 INFO:teuthology.orchestra.run.smithi167.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T23:40:31.580 INFO:teuthology.orchestra.run.smithi167.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T23:40:31.581 INFO:teuthology.orchestra.run.smithi012.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-13T23:40:31.581 INFO:teuthology.orchestra.run.smithi012.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-13T23:40:31.584 INFO:teuthology.orchestra.run.smithi012.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-13T23:40:31.589 INFO:teuthology.orchestra.run.smithi167.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T23:40:31.600 INFO:teuthology.orchestra.run.smithi167.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T23:40:31.600 INFO:teuthology.orchestra.run.smithi012.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-13T23:40:31.601 INFO:teuthology.orchestra.run.smithi012.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-13T23:40:31.601 INFO:teuthology.orchestra.run.smithi012.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-13T23:40:31.602 INFO:teuthology.orchestra.run.smithi012.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T23:40:31.603 INFO:teuthology.orchestra.run.smithi012.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-13T23:40:31.603 INFO:teuthology.orchestra.run.smithi012.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-13T23:40:31.605 INFO:teuthology.orchestra.run.smithi012.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-13T23:40:31.605 INFO:teuthology.orchestra.run.smithi167.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T23:40:31.605 INFO:teuthology.orchestra.run.smithi167.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T23:40:31.606 INFO:teuthology.orchestra.run.smithi012.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-13T23:40:31.606 INFO:teuthology.orchestra.run.smithi167.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T23:40:31.607 INFO:teuthology.orchestra.run.smithi167.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T23:40:31.607 INFO:teuthology.orchestra.run.smithi012.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-13T23:40:31.608 INFO:teuthology.orchestra.run.smithi167.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T23:40:31.610 INFO:teuthology.orchestra.run.smithi167.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T23:40:31.610 INFO:teuthology.orchestra.run.smithi167.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T23:40:31.612 INFO:teuthology.orchestra.run.smithi167.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T23:40:31.619 INFO:teuthology.orchestra.run.smithi012.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-13T23:40:31.623 INFO:teuthology.orchestra.run.smithi167.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T23:40:31.626 INFO:teuthology.orchestra.run.smithi012.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-13T23:40:31.630 INFO:teuthology.orchestra.run.smithi167.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T23:40:31.632 INFO:teuthology.orchestra.run.smithi167.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T23:40:31.632 INFO:teuthology.orchestra.run.smithi167.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T23:40:31.633 INFO:teuthology.orchestra.run.smithi012.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-13T23:40:31.633 INFO:teuthology.orchestra.run.smithi012.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-13T23:40:31.633 INFO:teuthology.orchestra.run.smithi167.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T23:40:31.633 INFO:teuthology.orchestra.run.smithi167.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T23:40:31.634 INFO:teuthology.orchestra.run.smithi012.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-13T23:40:31.634 INFO:teuthology.orchestra.run.smithi012.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-13T23:40:31.635 INFO:teuthology.orchestra.run.smithi167.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T23:40:31.635 INFO:teuthology.orchestra.run.smithi167.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T23:40:31.636 INFO:teuthology.orchestra.run.smithi167.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T23:40:31.636 INFO:teuthology.orchestra.run.smithi012.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-13T23:40:31.639 INFO:teuthology.orchestra.run.smithi012.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-13T23:40:31.639 INFO:teuthology.orchestra.run.smithi012.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-13T23:40:31.640 INFO:teuthology.orchestra.run.smithi167.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T23:40:31.641 INFO:teuthology.orchestra.run.smithi012.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-13T23:40:31.644 INFO:teuthology.orchestra.run.smithi167.stdout:Get:45 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-845-ge4e227b3-1focal [4,042 kB] 2024-09-13T23:40:31.653 INFO:teuthology.orchestra.run.smithi167.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T23:40:31.653 INFO:teuthology.orchestra.run.smithi012.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-asyncssh all 1.12.2-1 [152 kB] 2024-09-13T23:40:31.654 INFO:teuthology.orchestra.run.smithi167.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T23:40:31.655 INFO:teuthology.orchestra.run.smithi012.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-13T23:40:31.655 INFO:teuthology.orchestra.run.smithi167.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T23:40:31.656 INFO:teuthology.orchestra.run.smithi167.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T23:40:31.657 INFO:teuthology.orchestra.run.smithi012.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-13T23:40:31.657 INFO:teuthology.orchestra.run.smithi012.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-13T23:40:31.657 INFO:teuthology.orchestra.run.smithi012.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-13T23:40:31.658 INFO:teuthology.orchestra.run.smithi167.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T23:40:31.660 INFO:teuthology.orchestra.run.smithi167.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T23:40:31.661 INFO:teuthology.orchestra.run.smithi167.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T23:40:31.671 INFO:teuthology.orchestra.run.smithi012.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-13T23:40:31.672 INFO:teuthology.orchestra.run.smithi012.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-13T23:40:31.672 INFO:teuthology.orchestra.run.smithi012.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-13T23:40:31.673 INFO:teuthology.orchestra.run.smithi012.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-13T23:40:31.673 INFO:teuthology.orchestra.run.smithi012.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-13T23:40:31.688 INFO:teuthology.orchestra.run.smithi012.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T23:40:31.689 INFO:teuthology.orchestra.run.smithi012.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-13T23:40:31.691 INFO:teuthology.orchestra.run.smithi012.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-13T23:40:31.693 INFO:teuthology.orchestra.run.smithi012.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-13T23:40:31.695 INFO:teuthology.orchestra.run.smithi012.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-13T23:40:31.706 INFO:teuthology.orchestra.run.smithi012.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-13T23:40:31.706 INFO:teuthology.orchestra.run.smithi012.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-13T23:40:31.720 INFO:teuthology.orchestra.run.smithi167.stdout:Get:53 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-845-ge4e227b3-1focal [746 kB] 2024-09-13T23:40:31.732 INFO:teuthology.orchestra.run.smithi167.stdout:Get:54 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-845-ge4e227b3-1focal [387 kB] 2024-09-13T23:40:31.736 INFO:teuthology.orchestra.run.smithi012.stdout:Get:52 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 18.2.4-845-ge4e227b3-1focal [4,042 kB] 2024-09-13T23:40:31.788 INFO:teuthology.orchestra.run.smithi012.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T23:40:31.789 INFO:teuthology.orchestra.run.smithi167.stdout:Get:55 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-845-ge4e227b3-1focal [31.9 kB] 2024-09-13T23:40:31.789 INFO:teuthology.orchestra.run.smithi167.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-13T23:40:31.789 INFO:teuthology.orchestra.run.smithi167.stdout:Get:57 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-845-ge4e227b3-1focal [200 kB] 2024-09-13T23:40:31.793 INFO:teuthology.orchestra.run.smithi167.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-845-ge4e227b3-1focal [62.5 kB] 2024-09-13T23:40:31.794 INFO:teuthology.orchestra.run.smithi167.stdout:Get:59 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-845-ge4e227b3-1focal [368 kB] 2024-09-13T23:40:31.801 INFO:teuthology.orchestra.run.smithi167.stdout:Get:60 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-845-ge4e227b3-1focal [6,630 kB] 2024-09-13T23:40:31.805 INFO:teuthology.orchestra.run.smithi012.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T23:40:31.807 INFO:teuthology.orchestra.run.smithi012.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T23:40:31.811 INFO:teuthology.orchestra.run.smithi167.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-13T23:40:31.812 INFO:teuthology.orchestra.run.smithi167.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-13T23:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T23:40:31.818 INFO:teuthology.orchestra.run.smithi012.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T23:40:31.824 INFO:teuthology.orchestra.run.smithi167.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-13T23:40:31.824 INFO:teuthology.orchestra.run.smithi167.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-13T23:40:31.825 INFO:teuthology.orchestra.run.smithi167.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T23:40:31.825 INFO:teuthology.orchestra.run.smithi167.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T23:40:31.826 INFO:teuthology.orchestra.run.smithi167.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T23:40:31.826 INFO:teuthology.orchestra.run.smithi167.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T23:40:31.826 INFO:teuthology.orchestra.run.smithi167.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T23:40:31.828 INFO:teuthology.orchestra.run.smithi167.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T23:40:31.844 INFO:teuthology.orchestra.run.smithi167.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T23:40:31.845 INFO:teuthology.orchestra.run.smithi167.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T23:40:31.848 INFO:teuthology.orchestra.run.smithi167.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T23:40:31.850 INFO:teuthology.orchestra.run.smithi167.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T23:40:31.853 INFO:teuthology.orchestra.run.smithi167.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T23:40:31.853 INFO:teuthology.orchestra.run.smithi167.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T23:40:31.856 INFO:teuthology.orchestra.run.smithi167.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T23:40:31.858 INFO:teuthology.orchestra.run.smithi167.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T23:40:31.874 INFO:teuthology.orchestra.run.smithi167.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T23:40:31.877 INFO:teuthology.orchestra.run.smithi167.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T23:40:31.879 INFO:teuthology.orchestra.run.smithi167.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T23:40:31.881 INFO:teuthology.orchestra.run.smithi012.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-13T23:40:31.882 INFO:teuthology.orchestra.run.smithi012.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-13T23:40:31.883 INFO:teuthology.orchestra.run.smithi012.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-13T23:40:31.884 INFO:teuthology.orchestra.run.smithi012.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-13T23:40:31.885 INFO:teuthology.orchestra.run.smithi012.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-13T23:40:31.887 INFO:teuthology.orchestra.run.smithi012.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-13T23:40:31.887 INFO:teuthology.orchestra.run.smithi012.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-13T23:40:31.899 INFO:teuthology.orchestra.run.smithi012.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-13T23:40:31.901 INFO:teuthology.orchestra.run.smithi012.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-13T23:40:31.903 INFO:teuthology.orchestra.run.smithi012.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-13T23:40:31.916 INFO:teuthology.orchestra.run.smithi012.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-13T23:40:31.917 INFO:teuthology.orchestra.run.smithi012.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-13T23:40:31.917 INFO:teuthology.orchestra.run.smithi167.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T23:40:31.918 INFO:teuthology.orchestra.run.smithi167.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T23:40:31.918 INFO:teuthology.orchestra.run.smithi167.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T23:40:31.921 INFO:teuthology.orchestra.run.smithi012.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-13T23:40:31.925 INFO:teuthology.orchestra.run.smithi012.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-13T23:40:31.936 INFO:teuthology.orchestra.run.smithi012.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 18.2.4-845-ge4e227b3-1focal [746 kB] 2024-09-13T23:40:31.942 INFO:teuthology.orchestra.run.smithi012.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-13T23:40:31.944 INFO:teuthology.orchestra.run.smithi012.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-asyncssh-doc all 1.12.2-1 [258 kB] 2024-09-13T23:40:31.946 INFO:teuthology.orchestra.run.smithi012.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-13T23:40:31.975 INFO:teuthology.orchestra.run.smithi167.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T23:40:31.975 INFO:teuthology.orchestra.run.smithi167.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-845-ge4e227b3-1focal [113 kB] 2024-09-13T23:40:31.977 INFO:teuthology.orchestra.run.smithi167.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-845-ge4e227b3-1focal [478 kB] 2024-09-13T23:40:31.980 INFO:teuthology.orchestra.run.smithi012.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 18.2.4-845-ge4e227b3-1focal [387 kB] 2024-09-13T23:40:31.984 INFO:teuthology.orchestra.run.smithi167.stdout:Get:88 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-845-ge4e227b3-1focal [25.2 MB] 2024-09-13T23:40:31.989 INFO:teuthology.orchestra.run.smithi012.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-13T23:40:31.989 INFO:teuthology.orchestra.run.smithi012.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-13T23:40:31.990 INFO:teuthology.orchestra.run.smithi012.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-13T23:40:31.990 INFO:teuthology.orchestra.run.smithi012.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-13T23:40:31.992 INFO:teuthology.orchestra.run.smithi012.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 18.2.4-845-ge4e227b3-1focal [31.9 kB] 2024-09-13T23:40:31.993 INFO:teuthology.orchestra.run.smithi012.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 18.2.4-845-ge4e227b3-1focal [200 kB] 2024-09-13T23:40:31.995 INFO:teuthology.orchestra.run.smithi167.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T23:40:31.996 INFO:teuthology.orchestra.run.smithi167.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T23:40:31.997 INFO:teuthology.orchestra.run.smithi012.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 18.2.4-845-ge4e227b3-1focal [62.5 kB] 2024-09-13T23:40:31.997 INFO:teuthology.orchestra.run.smithi167.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T23:40:31.997 INFO:teuthology.orchestra.run.smithi167.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T23:40:31.998 INFO:teuthology.orchestra.run.smithi167.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T23:40:31.998 INFO:teuthology.orchestra.run.smithi012.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 18.2.4-845-ge4e227b3-1focal [368 kB] 2024-09-13T23:40:32.028 INFO:teuthology.orchestra.run.smithi012.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 18.2.4-845-ge4e227b3-1focal [6,630 kB] 2024-09-13T23:40:32.029 INFO:teuthology.orchestra.run.smithi012.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T23:40:32.029 INFO:teuthology.orchestra.run.smithi012.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-13T23:40:32.031 INFO:teuthology.orchestra.run.smithi012.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-13T23:40:32.031 INFO:teuthology.orchestra.run.smithi012.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-13T23:40:32.032 INFO:teuthology.orchestra.run.smithi012.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-13T23:40:32.348 INFO:teuthology.orchestra.run.smithi012.stdout:Get:91 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 18.2.4-845-ge4e227b3-1focal [113 kB] 2024-09-13T23:40:32.350 INFO:teuthology.orchestra.run.smithi012.stdout:Get:92 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 18.2.4-845-ge4e227b3-1focal [478 kB] 2024-09-13T23:40:32.357 INFO:teuthology.orchestra.run.smithi012.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 18.2.4-845-ge4e227b3-1focal [25.2 MB] 2024-09-13T23:40:32.725 INFO:teuthology.orchestra.run.smithi167.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-845-ge4e227b3-1focal [9,611 kB] 2024-09-13T23:40:32.914 INFO:teuthology.orchestra.run.smithi167.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-845-ge4e227b3-1focal [238 kB] 2024-09-13T23:40:32.923 INFO:teuthology.orchestra.run.smithi167.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-845-ge4e227b3-1focal [148 kB] 2024-09-13T23:40:32.925 INFO:teuthology.orchestra.run.smithi167.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-845-ge4e227b3-1focal [1,574 kB] 2024-09-13T23:40:32.973 INFO:teuthology.orchestra.run.smithi167.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-845-ge4e227b3-1focal [6,640 kB] 2024-09-13T23:40:33.164 INFO:teuthology.orchestra.run.smithi167.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-845-ge4e227b3-1focal [24.5 MB] 2024-09-13T23:40:33.347 INFO:teuthology.orchestra.run.smithi012.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 18.2.4-845-ge4e227b3-1focal [9,611 kB] 2024-09-13T23:40:33.668 INFO:teuthology.orchestra.run.smithi012.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 18.2.4-845-ge4e227b3-1focal [238 kB] 2024-09-13T23:40:33.672 INFO:teuthology.orchestra.run.smithi012.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 18.2.4-845-ge4e227b3-1focal [148 kB] 2024-09-13T23:40:33.674 INFO:teuthology.orchestra.run.smithi012.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 18.2.4-845-ge4e227b3-1focal [1,574 kB] 2024-09-13T23:40:33.731 INFO:teuthology.orchestra.run.smithi012.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 18.2.4-845-ge4e227b3-1focal [6,640 kB] 2024-09-13T23:40:33.942 INFO:teuthology.orchestra.run.smithi167.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-845-ge4e227b3-1focal [13.7 kB] 2024-09-13T23:40:33.942 INFO:teuthology.orchestra.run.smithi167.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-845-ge4e227b3-1focal [856 kB] 2024-09-13T23:40:33.953 INFO:teuthology.orchestra.run.smithi167.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-845-ge4e227b3-1focal [2,295 kB] 2024-09-13T23:40:33.968 INFO:teuthology.orchestra.run.smithi012.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 18.2.4-845-ge4e227b3-1focal [24.5 MB] 2024-09-13T23:40:33.970 INFO:teuthology.orchestra.run.smithi167.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-845-ge4e227b3-1focal [239 kB] 2024-09-13T23:40:33.972 INFO:teuthology.orchestra.run.smithi167.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-845-ge4e227b3-1focal [143 kB] 2024-09-13T23:40:33.973 INFO:teuthology.orchestra.run.smithi167.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-845-ge4e227b3-1focal [7,015 kB] 2024-09-13T23:40:34.164 INFO:teuthology.orchestra.run.smithi167.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-845-ge4e227b3-1focal [7,882 kB] 2024-09-13T23:40:34.389 INFO:teuthology.orchestra.run.smithi167.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-845-ge4e227b3-1focal [13.9 kB] 2024-09-13T23:40:34.389 INFO:teuthology.orchestra.run.smithi167.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-845-ge4e227b3-1focal [46.9 MB] 2024-09-13T23:40:35.029 INFO:teuthology.orchestra.run.smithi012.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 18.2.4-845-ge4e227b3-1focal [13.7 kB] 2024-09-13T23:40:35.029 INFO:teuthology.orchestra.run.smithi012.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 18.2.4-845-ge4e227b3-1focal [856 kB] 2024-09-13T23:40:35.043 INFO:teuthology.orchestra.run.smithi012.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 18.2.4-845-ge4e227b3-1focal [2,295 kB] 2024-09-13T23:40:35.082 INFO:teuthology.orchestra.run.smithi012.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 18.2.4-845-ge4e227b3-1focal [239 kB] 2024-09-13T23:40:35.087 INFO:teuthology.orchestra.run.smithi012.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 18.2.4-845-ge4e227b3-1focal [143 kB] 2024-09-13T23:40:35.090 INFO:teuthology.orchestra.run.smithi012.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 18.2.4-845-ge4e227b3-1focal [7,015 kB] 2024-09-13T23:40:35.367 INFO:teuthology.orchestra.run.smithi012.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 18.2.4-845-ge4e227b3-1focal [7,882 kB] 2024-09-13T23:40:35.693 INFO:teuthology.orchestra.run.smithi012.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 18.2.4-845-ge4e227b3-1focal [13.9 kB] 2024-09-13T23:40:35.693 INFO:teuthology.orchestra.run.smithi012.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 18.2.4-845-ge4e227b3-1focal [46.9 MB] 2024-09-13T23:40:35.910 INFO:teuthology.orchestra.run.smithi167.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-845-ge4e227b3-1focal [120 kB] 2024-09-13T23:40:35.912 INFO:teuthology.orchestra.run.smithi167.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-845-ge4e227b3-1focal [37.4 kB] 2024-09-13T23:40:35.913 INFO:teuthology.orchestra.run.smithi167.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-845-ge4e227b3-1focal [13.5 MB] 2024-09-13T23:40:36.208 INFO:teuthology.orchestra.run.smithi167.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-845-ge4e227b3-1focal [107 kB] 2024-09-13T23:40:36.857 INFO:teuthology.orchestra.run.smithi167.stdout:Fetched 199 MB in 5s (40.7 MB/s) 2024-09-13T23:40:36.858 INFO:teuthology.orchestra.run.smithi012.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 18.2.4-845-ge4e227b3-1focal [120 kB] 2024-09-13T23:40:36.861 INFO:teuthology.orchestra.run.smithi012.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 18.2.4-845-ge4e227b3-1focal [37.4 kB] 2024-09-13T23:40:36.862 INFO:teuthology.orchestra.run.smithi012.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 18.2.4-845-ge4e227b3-1focal [13.5 MB] 2024-09-13T23:40:36.992 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T23:40:37.037 INFO:teuthology.orchestra.run.smithi012.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/reef/e4e227b37858f5109138494e01e5ba1ba7b8ffe8/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 18.2.4-845-ge4e227b3-1focal [107 kB] 2024-09-13T23:40:37.050 INFO:teuthology.orchestra.run.smithi167.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-13T23:40:37.054 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T23:40:37.124 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T23:40:37.469 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T23:40:37.485 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T23:40:37.527 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T23:40:37.705 INFO:teuthology.orchestra.run.smithi012.stdout:Fetched 199 MB in 6s (35.0 MB/s) 2024-09-13T23:40:37.897 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T23:40:37.907 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-13T23:40:37.913 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../002-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T23:40:37.955 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T23:40:37.962 INFO:teuthology.orchestra.run.smithi012.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-13T23:40:37.967 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-13T23:40:38.047 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T23:40:38.326 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T23:40:38.344 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../003-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T23:40:38.384 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T23:40:38.385 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T23:40:38.401 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-13T23:40:38.442 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T23:40:38.795 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T23:40:38.812 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../002-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-13T23:40:38.853 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T23:40:38.904 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T23:40:38.919 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../004-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T23:40:38.962 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T23:40:39.308 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T23:40:39.315 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T23:40:39.327 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../003-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T23:40:39.331 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../005-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T23:40:39.374 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T23:40:39.375 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T23:40:39.735 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T23:40:39.750 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../006-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T23:40:39.785 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T23:40:39.860 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T23:40:39.879 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../004-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T23:40:39.918 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T23:40:40.124 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../007-librbd1_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:40.247 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librbd1 (18.2.4-845-ge4e227b3-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T23:40:40.304 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T23:40:40.321 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../005-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-13T23:40:40.363 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T23:40:40.766 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-13T23:40:40.784 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../006-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-13T23:40:40.824 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T23:40:40.944 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../008-librados2_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:41.068 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librados2 (18.2.4-845-ge4e227b3-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T23:40:41.334 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../007-librbd1_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:41.461 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librbd1 (18.2.4-845-ge4e227b3-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T23:40:41.772 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T23:40:41.790 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../009-libcephfs2_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:41.830 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libcephfs2 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:42.158 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-rados. 2024-09-13T23:40:42.174 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../010-python3-rados_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:42.183 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../008-librados2_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:42.216 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-rados (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:42.307 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librados2 (18.2.4-845-ge4e227b3-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-13T23:40:42.594 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T23:40:42.610 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:40:42.652 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-ceph-argparse (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:42.880 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T23:40:42.897 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../012-python3-cephfs_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:42.947 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cephfs (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:43.018 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T23:40:43.039 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../009-libcephfs2_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:43.077 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs2 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:43.283 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T23:40:43.298 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../013-python3-ceph-common_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:40:43.341 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-ceph-common (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:43.413 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rados. 2024-09-13T23:40:43.429 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../010-python3-rados_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:43.471 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rados (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:43.669 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T23:40:43.688 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../014-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T23:40:43.727 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T23:40:43.857 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T23:40:43.873 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../011-python3-ceph-argparse_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:40:43.915 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-argparse (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:43.980 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T23:40:43.996 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../015-python3-rbd_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:44.038 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-rbd (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:44.168 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T23:40:44.184 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../012-python3-cephfs_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:44.226 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cephfs (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:44.383 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T23:40:44.401 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../016-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T23:40:44.433 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T23:40:44.570 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T23:40:44.590 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../013-python3-ceph-common_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:40:44.628 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-ceph-common (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:44.777 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T23:40:44.796 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../017-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T23:40:44.836 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T23:40:44.932 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T23:40:44.951 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../014-python3-prettytable_0.7.2-5_all.deb ... 2024-09-13T23:40:44.990 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-13T23:40:45.172 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T23:40:45.188 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../018-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T23:40:45.231 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T23:40:45.259 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T23:40:45.279 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../015-python3-rbd_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:45.309 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rbd (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:45.575 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package librgw2. 2024-09-13T23:40:45.591 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../019-librgw2_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:45.625 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking librgw2 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:45.645 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-13T23:40:45.661 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../016-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-13T23:40:45.703 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T23:40:46.089 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-13T23:40:46.105 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../017-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-13T23:40:46.148 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T23:40:46.484 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T23:40:46.501 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../018-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-13T23:40:46.542 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T23:40:46.736 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T23:40:46.752 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:46.803 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-rgw (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:46.903 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package librgw2. 2024-09-13T23:40:46.920 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../019-librgw2_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:46.961 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking librgw2 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:47.156 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liboath0. 2024-09-13T23:40:47.171 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T23:40:47.214 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T23:40:47.483 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T23:40:47.499 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:47.534 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libradosstriper1 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:47.864 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T23:40:47.880 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../020-python3-rgw_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:47.922 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rgw (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:47.987 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-common. 2024-09-13T23:40:48.003 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../023-ceph-common_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:48.045 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-common (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:48.291 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liboath0. 2024-09-13T23:40:48.308 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-13T23:40:48.352 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-13T23:40:48.694 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T23:40:48.710 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../022-libradosstriper1_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:48.752 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libradosstriper1 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:49.564 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-common. 2024-09-13T23:40:49.581 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../023-ceph-common_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:49.622 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-common (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:50.493 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-base. 2024-09-13T23:40:50.510 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../024-ceph-base_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:50.561 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-base (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:51.773 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T23:40:51.790 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T23:40:51.830 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T23:40:52.193 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T23:40:52.210 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../026-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T23:40:52.251 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T23:40:52.285 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-base. 2024-09-13T23:40:52.304 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../024-ceph-base_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:52.354 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-base (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:52.571 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T23:40:52.589 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T23:40:52.629 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T23:40:52.932 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T23:40:52.952 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T23:40:52.990 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T23:40:53.335 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-mako. 2024-09-13T23:40:53.351 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T23:40:53.393 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T23:40:53.590 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T23:40:53.606 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../025-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-13T23:40:53.648 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-13T23:40:53.713 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T23:40:53.730 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T23:40:53.771 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T23:40:53.993 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T23:40:54.009 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../026-python3-natsort_7.0.1-1_all.deb ... 2024-09-13T23:40:54.043 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-13T23:40:54.091 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T23:40:54.108 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T23:40:54.149 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T23:40:54.379 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T23:40:54.396 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../027-python3-logutils_0.3.3-7_all.deb ... 2024-09-13T23:40:54.437 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-13T23:40:54.452 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-webob. 2024-09-13T23:40:54.467 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T23:40:54.510 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T23:40:54.748 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T23:40:54.765 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../028-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-13T23:40:54.806 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-13T23:40:54.838 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T23:40:54.854 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T23:40:54.896 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T23:40:55.134 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-mako. 2024-09-13T23:40:55.153 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../029-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-13T23:40:55.192 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T23:40:55.216 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T23:40:55.232 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T23:40:55.274 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T23:40:55.521 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T23:40:55.537 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../030-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T23:40:55.579 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T23:40:55.628 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T23:40:55.646 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T23:40:55.680 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T23:40:55.865 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T23:40:55.883 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../031-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-13T23:40:55.948 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T23:40:56.023 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T23:40:56.039 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T23:40:56.081 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T23:40:56.251 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webob. 2024-09-13T23:40:56.268 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../032-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-13T23:40:56.309 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T23:40:56.350 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-paste. 2024-09-13T23:40:56.367 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T23:40:56.409 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T23:40:56.629 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-13T23:40:56.645 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../033-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-13T23:40:56.687 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T23:40:56.746 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T23:40:56.762 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T23:40:56.804 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T23:40:57.015 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-bs4. 2024-09-13T23:40:57.032 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../034-python3-bs4_4.8.2-1_all.deb ... 2024-09-13T23:40:57.073 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-13T23:40:57.132 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T23:40:57.151 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T23:40:57.190 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T23:40:57.426 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T23:40:57.442 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../035-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-13T23:40:57.487 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T23:40:57.493 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T23:40:57.513 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T23:40:57.551 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T23:40:57.837 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T23:40:57.853 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../036-python3-tempita_0.5.2-6_all.deb ... 2024-09-13T23:40:57.871 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T23:40:57.888 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T23:40:57.895 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-13T23:40:57.937 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T23:40:58.173 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-paste. 2024-09-13T23:40:58.191 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../037-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-13T23:40:58.231 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T23:40:58.284 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T23:40:58.304 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T23:40:58.342 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T23:40:58.576 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T23:40:58.593 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../038-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-13T23:40:58.634 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T23:40:58.637 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T23:40:58.657 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../043-ceph-mgr-modules-core_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:40:58.695 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:58.904 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T23:40:58.920 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../039-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-13T23:40:58.953 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-13T23:40:59.092 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T23:40:59.112 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../044-libsqlite3-mod-ceph_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:40:59.150 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:40:59.298 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T23:40:59.316 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../040-python3-webtest_2.0.32-2_all.deb ... 2024-09-13T23:40:59.356 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-13T23:40:59.436 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T23:40:59.456 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../045-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T23:40:59.495 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T23:40:59.664 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T23:40:59.683 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../041-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-13T23:40:59.723 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T23:40:59.822 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T23:40:59.841 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../046-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T23:40:59.890 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T23:41:00.095 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T23:41:00.110 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../042-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-13T23:41:00.153 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T23:41:00.262 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T23:41:00.281 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../047-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T23:41:00.322 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T23:41:00.447 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T23:41:00.465 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../043-ceph-mgr-modules-core_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:41:00.497 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:00.634 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T23:41:00.650 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../048-ceph-mgr_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:00.692 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:00.899 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T23:41:00.919 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../044-libsqlite3-mod-ceph_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:00.968 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:01.187 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T23:41:01.203 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../049-ceph-mon_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:01.245 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mon (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:01.280 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T23:41:01.299 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../045-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-13T23:41:01.338 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T23:41:01.643 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cryptography. 2024-09-13T23:41:01.660 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../046-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-13T23:41:01.701 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T23:41:02.063 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T23:41:02.079 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../047-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-13T23:41:02.121 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-13T23:41:02.428 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T23:41:02.447 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../048-ceph-mgr_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:02.478 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:02.956 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T23:41:02.973 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../049-ceph-mon_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:03.014 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mon (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:03.033 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T23:41:03.048 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../050-ceph-osd_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:03.082 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-osd (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:03.942 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T23:41:03.958 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../050-ceph-osd_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:04.000 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-osd (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:05.269 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph. 2024-09-13T23:41:05.286 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../051-ceph_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:05.327 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:05.680 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T23:41:05.695 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../052-ceph-fuse_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:05.738 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-fuse (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:06.152 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T23:41:06.168 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../053-ceph-mds_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:06.210 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mds (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:06.278 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph. 2024-09-13T23:41:06.296 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../051-ceph_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:06.336 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:06.663 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T23:41:06.679 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../052-ceph-fuse_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:06.721 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-fuse (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:06.755 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package cephadm. 2024-09-13T23:41:06.771 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../054-cephadm_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:06.813 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking cephadm (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:07.108 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T23:41:07.117 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T23:41:07.124 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../055-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T23:41:07.136 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../053-ceph-mds_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:07.158 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T23:41:07.175 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mds (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:07.553 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T23:41:07.573 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T23:41:07.611 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T23:41:07.711 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package cephadm. 2024-09-13T23:41:07.728 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../054-cephadm_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:07.761 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking cephadm (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:07.913 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T23:41:07.933 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:41:08.022 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:08.055 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T23:41:08.072 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../055-python3-asyncssh_1.12.2-1_all.deb ... 2024-09-13T23:41:08.113 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-asyncssh (1.12.2-1) ... 2024-09-13T23:41:08.375 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T23:41:08.391 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../058-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T23:41:08.433 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T23:41:08.508 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T23:41:08.525 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-13T23:41:08.566 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T23:41:08.761 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-attr. 2024-09-13T23:41:08.779 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../059-python3-attr_19.3.0-2_all.deb ... 2024-09-13T23:41:08.819 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T23:41:08.894 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T23:41:08.913 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:41:08.952 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:09.139 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T23:41:09.158 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../060-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T23:41:09.197 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T23:41:09.448 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-jwt. 2024-09-13T23:41:09.467 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../058-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-13T23:41:09.505 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T23:41:09.533 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-py. 2024-09-13T23:41:09.551 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../061-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T23:41:09.591 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T23:41:09.808 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-attr. 2024-09-13T23:41:09.827 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../059-python3-attr_19.3.0-2_all.deb ... 2024-09-13T23:41:09.866 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-13T23:41:09.920 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T23:41:09.936 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../062-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T23:41:09.978 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T23:41:10.227 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T23:41:10.245 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../060-python3-pluggy_0.13.0-2_all.deb ... 2024-09-13T23:41:10.281 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T23:41:10.285 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-13T23:41:10.300 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../063-python3-packaging_20.3-1_all.deb ... 2024-09-13T23:41:10.339 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T23:41:10.647 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-py. 2024-09-13T23:41:10.658 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T23:41:10.662 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../061-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-13T23:41:10.680 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../064-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T23:41:10.716 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T23:41:10.721 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T23:41:11.020 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T23:41:11.037 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../065-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T23:41:11.058 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-13T23:41:11.074 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../062-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-13T23:41:11.078 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T23:41:11.116 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T23:41:11.419 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-packaging. 2024-09-13T23:41:11.435 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../063-python3-packaging_20.3-1_all.deb ... 2024-09-13T23:41:11.440 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T23:41:11.456 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../066-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T23:41:11.477 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-13T23:41:11.498 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T23:41:11.751 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T23:41:11.767 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../067-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T23:41:11.804 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T23:41:11.809 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T23:41:11.821 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../064-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-13T23:41:11.863 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T23:41:12.103 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T23:41:12.122 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../068-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T23:41:12.161 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T23:41:12.165 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T23:41:12.181 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../065-python3-pytest_4.6.9-1_all.deb ... 2024-09-13T23:41:12.224 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-13T23:41:12.489 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-routes. 2024-09-13T23:41:12.509 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../069-python3-routes_2.4.1-2_all.deb ... 2024-09-13T23:41:12.547 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T23:41:12.604 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T23:41:12.621 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../066-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T23:41:12.661 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T23:41:12.876 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T23:41:12.894 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:41:12.897 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjs-underscore. 2024-09-13T23:41:12.914 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../067-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-13T23:41:12.933 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:12.955 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T23:41:13.233 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-13T23:41:13.252 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../068-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-13T23:41:13.291 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T23:41:13.636 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-routes. 2024-09-13T23:41:13.653 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../069-python3-routes_2.4.1-2_all.deb ... 2024-09-13T23:41:13.693 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-13T23:41:14.022 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T23:41:14.038 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../070-ceph-mgr-dashboard_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:41:14.080 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:14.389 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T23:41:14.407 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../071-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T23:41:14.447 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T23:41:14.758 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T23:41:14.775 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../072-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T23:41:14.816 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T23:41:15.119 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T23:41:15.138 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../073-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T23:41:15.177 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T23:41:15.477 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-decorator. 2024-09-13T23:41:15.494 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../071-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-13T23:41:15.535 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T23:41:15.829 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-13T23:41:15.848 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../072-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-13T23:41:15.887 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T23:41:16.215 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-scipy. 2024-09-13T23:41:16.235 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../073-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-13T23:41:16.273 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-13T23:41:16.486 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T23:41:16.503 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T23:41:16.544 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T23:41:17.081 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T23:41:17.099 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../075-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T23:41:17.172 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T23:41:17.526 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T23:41:17.545 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../076-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T23:41:17.548 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-13T23:41:17.565 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-13T23:41:17.584 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T23:41:17.606 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T23:41:18.143 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T23:41:18.159 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../075-python3-joblib_0.14.0-3_all.deb ... 2024-09-13T23:41:18.201 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-13T23:41:18.605 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T23:41:18.622 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../076-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-13T23:41:18.660 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T23:41:18.662 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T23:41:18.677 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:41:18.718 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:19.405 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T23:41:19.422 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../077-ceph-mgr-diskprediction-local_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:41:19.463 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:19.821 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T23:41:19.839 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../078-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T23:41:19.879 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T23:41:20.241 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T23:41:20.259 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T23:41:20.299 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T23:41:20.600 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T23:41:20.619 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T23:41:20.620 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../078-python3-cachetools_4.0.0-1_all.deb ... 2024-09-13T23:41:20.636 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T23:41:20.677 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T23:41:21.008 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-13T23:41:21.112 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T23:41:21.128 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../081-python3-rsa_4.0-3_all.deb ... 2024-09-13T23:41:21.161 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T23:41:21.311 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T23:41:21.328 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../079-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-13T23:41:21.369 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T23:41:21.482 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T23:41:21.501 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T23:41:21.540 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T23:41:21.739 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T23:41:21.759 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../080-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-13T23:41:21.797 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T23:41:21.886 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T23:41:21.903 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../083-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T23:41:21.960 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T23:41:22.202 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T23:41:22.219 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../081-python3-rsa_4.0-3_all.deb ... 2024-09-13T23:41:22.254 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-13T23:41:22.264 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T23:41:22.280 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../084-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T23:41:22.321 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T23:41:22.576 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T23:41:22.596 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../082-python3-google-auth_1.5.1-2_all.deb ... 2024-09-13T23:41:22.634 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-13T23:41:22.651 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T23:41:22.668 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../085-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T23:41:22.709 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T23:41:22.955 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-blinker. 2024-09-13T23:41:22.973 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../083-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-13T23:41:23.013 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T23:41:23.046 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T23:41:23.063 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../086-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T23:41:23.104 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T23:41:23.300 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-13T23:41:23.317 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../084-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-13T23:41:23.383 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T23:41:23.419 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T23:41:23.436 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../087-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T23:41:23.477 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T23:41:23.728 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T23:41:23.745 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../085-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-13T23:41:23.786 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T23:41:24.024 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T23:41:24.043 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../088-ceph-mgr-k8sevents_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:41:24.082 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:24.098 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T23:41:24.115 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../086-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-13T23:41:24.156 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T23:41:24.419 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T23:41:24.436 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../089-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T23:41:24.477 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T23:41:24.501 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T23:41:24.520 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../087-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-13T23:41:24.559 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T23:41:24.866 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T23:41:24.884 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../090-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T23:41:24.932 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T23:41:25.160 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T23:41:25.178 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../088-ceph-mgr-k8sevents_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:41:25.202 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package jq. 2024-09-13T23:41:25.217 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:25.218 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../091-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T23:41:25.260 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T23:41:25.496 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package socat. 2024-09-13T23:41:25.514 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../092-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T23:41:25.522 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T23:41:25.540 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../089-libonig5_6.9.4-1_amd64.deb ... 2024-09-13T23:41:25.554 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T23:41:25.580 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-13T23:41:25.890 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T23:41:25.910 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../093-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T23:41:25.948 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T23:41:25.959 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T23:41:25.975 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../090-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T23:41:26.017 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T23:41:26.278 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package jq. 2024-09-13T23:41:26.295 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-test. 2024-09-13T23:41:26.296 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../091-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-13T23:41:26.314 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../094-ceph-test_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:26.336 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T23:41:26.352 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-test (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:26.588 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package socat. 2024-09-13T23:41:26.608 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../092-socat_1.7.3.3-2_amd64.deb ... 2024-09-13T23:41:26.655 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-13T23:41:26.975 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T23:41:26.995 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../093-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-13T23:41:27.041 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-13T23:41:27.379 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-test. 2024-09-13T23:41:27.397 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../094-ceph-test_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:27.453 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-test (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:30.457 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T23:41:30.475 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../095-ceph-volume_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:41:30.515 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking ceph-volume (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:30.835 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T23:41:30.852 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T23:41:30.893 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T23:41:31.364 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T23:41:31.382 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../097-libcephfs-dev_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:31.422 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libcephfs-dev (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:31.774 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T23:41:31.790 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../095-ceph-volume_18.2.4-845-ge4e227b3-1focal_all.deb ... 2024-09-13T23:41:31.825 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T23:41:31.840 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking ceph-volume (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:31.843 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T23:41:31.902 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T23:41:32.199 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T23:41:32.218 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T23:41:32.255 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-13T23:41:32.257 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T23:41:32.275 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../096-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T23:41:32.310 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T23:41:32.669 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T23:41:32.685 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T23:41:32.727 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T23:41:32.789 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T23:41:32.806 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../097-libcephfs-dev_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:32.847 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libcephfs-dev (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:33.268 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T23:41:33.285 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../098-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-13T23:41:33.343 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T23:41:33.683 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T23:41:33.700 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../099-python-asyncssh-doc_1.12.2-1_all.deb ... 2024-09-13T23:41:33.741 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-asyncssh-doc (1.12.2-1) ... 2024-09-13T23:41:34.161 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-13T23:41:34.180 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../100-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-13T23:41:34.219 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T23:41:34.310 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T23:41:34.331 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T23:41:34.368 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T23:41:34.740 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T23:41:34.757 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T23:41:34.798 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T23:41:35.101 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T23:41:35.122 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T23:41:35.159 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T23:41:35.512 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T23:41:35.529 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T23:41:35.570 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T23:41:35.796 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-13T23:41:35.815 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../101-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-13T23:41:35.854 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-13T23:41:36.024 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T23:41:36.043 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../105-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T23:41:36.081 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T23:41:36.191 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-webencodings. 2024-09-13T23:41:36.211 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../102-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-13T23:41:36.249 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T23:41:36.435 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T23:41:36.453 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../106-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T23:41:36.493 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T23:41:36.560 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-html5lib. 2024-09-13T23:41:36.580 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../103-python3-html5lib_1.0.1-2_all.deb ... 2024-09-13T23:41:36.609 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-13T23:41:36.816 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T23:41:36.836 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T23:41:36.874 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T23:41:36.955 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-13T23:41:36.973 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../104-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-13T23:41:37.012 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T23:41:37.177 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T23:41:37.197 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../108-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T23:41:37.235 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T23:41:37.500 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T23:41:37.522 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../105-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T23:41:37.538 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T23:41:37.673 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T23:41:37.673 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../109-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T23:41:37.673 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T23:41:37.878 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T23:41:37.896 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../106-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-13T23:41:37.936 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T23:41:38.242 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package radosgw. 2024-09-13T23:41:38.260 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../110-radosgw_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:38.298 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T23:41:38.315 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-13T23:41:38.318 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking radosgw (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:38.358 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T23:41:38.661 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T23:41:38.678 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../108-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-13T23:41:38.728 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T23:41:39.014 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T23:41:39.031 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../109-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-13T23:41:39.072 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T23:41:39.720 INFO:teuthology.orchestra.run.smithi167.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T23:41:39.739 INFO:teuthology.orchestra.run.smithi167.stdout:Preparing to unpack .../111-rbd-fuse_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:39.742 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package radosgw. 2024-09-13T23:41:39.759 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../110-radosgw_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:39.778 INFO:teuthology.orchestra.run.smithi167.stdout:Unpacking rbd-fuse (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:39.800 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking radosgw (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:40.076 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T23:41:40.369 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T23:41:40.495 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T23:41:40.621 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T23:41:40.814 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up cephadm (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:41.196 INFO:teuthology.orchestra.run.smithi012.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T23:41:41.213 INFO:teuthology.orchestra.run.smithi012.stdout:Preparing to unpack .../111-rbd-fuse_18.2.4-845-ge4e227b3-1focal_amd64.deb ... 2024-09-13T23:41:41.253 INFO:teuthology.orchestra.run.smithi012.stdout:Unpacking rbd-fuse (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:41.349 INFO:teuthology.orchestra.run.smithi167.stdout:Adding system user cephadm....done 2024-09-13T23:41:41.514 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T23:41:41.596 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-13T23:41:41.640 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T23:41:41.911 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-13T23:41:41.991 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T23:41:42.038 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-13T23:41:42.189 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-13T23:41:42.276 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T23:41:42.390 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up cephadm (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:42.402 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T23:41:42.528 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T23:41:42.855 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T23:41:42.941 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user cephadm....done 2024-09-13T23:41:43.131 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-13T23:41:43.257 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-13T23:41:43.265 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T23:41:43.583 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T23:41:43.626 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T23:41:43.868 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T23:41:43.910 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-13T23:41:44.036 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-13T23:41:44.144 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T23:41:44.162 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-13T23:41:44.428 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-ceph-argparse (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:44.481 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-13T23:41:44.721 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T23:41:44.847 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T23:41:44.882 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-13T23:41:45.115 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T23:41:45.176 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-13T23:41:45.408 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T23:41:45.469 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-13T23:41:45.534 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T23:41:45.753 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-13T23:41:45.902 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T23:41:46.029 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-argparse (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:46.219 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T23:41:46.330 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-13T23:41:46.473 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-13T23:41:46.504 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T23:41:46.765 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-13T23:41:46.847 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T23:41:47.050 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-13T23:41:47.123 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T23:41:47.201 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-13T23:41:47.449 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T23:41:47.569 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-13T23:41:47.734 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T23:41:47.870 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-13T23:41:48.035 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T23:41:48.161 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T23:41:48.162 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-13T23:41:48.287 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T23:41:48.522 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-13T23:41:48.580 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T23:41:48.806 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-13T23:41:48.981 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T23:41:49.141 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-13T23:41:49.434 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-13T23:41:49.533 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T23:41:49.726 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-13T23:41:49.861 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-13T23:41:49.893 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T23:41:49.987 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-13T23:41:50.019 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-ceph-common (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:50.272 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-13T23:41:50.354 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T23:41:50.488 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T23:41:50.681 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-13T23:41:50.840 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T23:41:50.966 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T23:41:51.050 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-13T23:41:51.092 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T23:41:51.384 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T23:41:51.418 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-13T23:41:51.544 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-ceph-common (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:41:51.694 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T23:41:51.845 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T23:41:51.971 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T23:41:52.221 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T23:41:52.298 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T23:41:52.355 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-13T23:41:52.574 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T23:41:52.698 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-13T23:41:52.824 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-13T23:41:52.867 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T23:41:52.975 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-13T23:41:52.992 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T23:41:53.269 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T23:41:53.276 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-13T23:41:53.561 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T23:41:53.561 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T23:41:53.687 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-13T23:41:53.829 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-13T23:41:53.854 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T23:41:54.164 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-13T23:41:54.242 INFO:teuthology.orchestra.run.smithi167.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T23:41:54.449 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-13T23:41:54.490 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T23:41:54.608 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T23:41:54.741 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-13T23:41:54.867 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T23:41:54.951 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T23:41:55.144 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-13T23:41:55.327 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T23:41:55.428 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-13T23:41:55.453 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T23:41:55.602 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T23:41:55.746 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-13T23:41:55.939 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T23:41:56.172 INFO:teuthology.orchestra.run.smithi012.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-13T23:41:56.232 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T23:41:56.415 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-13T23:41:56.541 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-13T23:41:56.859 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-13T23:41:57.227 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-13T23:41:57.353 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-13T23:41:57.479 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-13T23:41:57.679 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T23:41:57.814 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T23:41:57.830 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-13T23:41:58.165 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-13T23:41:58.182 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T23:41:58.500 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T23:41:58.793 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T23:41:58.919 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T23:41:59.253 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T23:41:59.571 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T23:41:59.687 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-13T23:41:59.813 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-13T23:41:59.956 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T23:42:00.198 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-13T23:42:00.274 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T23:42:00.490 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-13T23:42:00.775 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-13T23:42:00.901 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-13T23:42:01.277 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-13T23:42:01.321 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T23:42:01.481 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T23:42:01.607 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T23:42:01.612 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-asyncssh (1.12.2-1) ... 2024-09-13T23:42:01.733 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T23:42:02.019 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-13T23:42:02.034 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T23:42:02.160 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T23:42:02.357 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-13T23:42:02.461 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T23:42:02.754 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T23:42:03.056 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T23:42:03.416 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librados2 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:03.429 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-13T23:42:03.542 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T23:42:03.555 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-13T23:42:03.668 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librgw2 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:03.706 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-13T23:42:03.794 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:03.823 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-13T23:42:04.006 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libcephfs2 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:04.108 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-13T23:42:04.132 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T23:42:04.259 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-13T23:42:04.467 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libradosstriper1 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:04.577 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-13T23:42:04.593 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T23:42:04.878 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T23:42:04.886 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up librbd1 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:05.012 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:05.138 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-fuse (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:05.213 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-13T23:42:05.400 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T23:42:05.400 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T23:42:05.606 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librados2 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:05.732 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python-asyncssh-doc (1.12.2-1) ... 2024-09-13T23:42:05.857 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up libcephfs-dev (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:05.858 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librgw2 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:05.984 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-rados (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:05.985 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libsqlite3-mod-ceph (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:06.109 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T23:42:06.221 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs2 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:06.347 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-13T23:42:06.640 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libradosstriper1 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:06.766 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-13T23:42:07.079 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-rbd (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:07.085 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up librbd1 (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:07.229 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-modules-core (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:07.230 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up rbd-fuse (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:07.353 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-fuse (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:07.357 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-rgw (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:07.483 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up python3-cephfs (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:07.581 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T23:42:07.582 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T23:42:07.609 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-common (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:08.063 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up libcephfs-dev (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:08.177 INFO:teuthology.orchestra.run.smithi167.stdout:Adding group ceph....done 2024-09-13T23:42:08.189 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rados (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:08.315 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-13T23:42:08.555 INFO:teuthology.orchestra.run.smithi167.stdout:Adding system user ceph....done 2024-09-13T23:42:08.723 INFO:teuthology.orchestra.run.smithi167.stdout:Setting system user ceph properties....done 2024-09-13T23:42:08.748 INFO:teuthology.orchestra.run.smithi167.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T23:42:08.899 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T23:42:09.213 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T23:42:09.245 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rbd (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:09.395 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up rbd-fuse (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:09.522 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-rgw (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:09.647 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up python3-cephfs (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:09.694 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-test (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:09.774 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-common (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:09.820 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up radosgw (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:10.299 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T23:42:10.300 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T23:42:10.352 INFO:teuthology.orchestra.run.smithi012.stdout:Adding group ceph....done 2024-09-13T23:42:10.703 INFO:teuthology.orchestra.run.smithi012.stdout:Adding system user ceph....done 2024-09-13T23:42:10.714 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-base (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:10.871 INFO:teuthology.orchestra.run.smithi012.stdout:Setting system user ceph properties....done 2024-09-13T23:42:10.891 INFO:teuthology.orchestra.run.smithi012.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T23:42:11.036 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T23:42:11.101 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T23:42:11.342 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T23:42:11.541 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mds (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:11.769 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T23:42:11.769 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T23:42:11.842 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-test (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:11.968 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up radosgw (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:12.209 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:12.418 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T23:42:12.419 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T23:42:12.461 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T23:42:12.461 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T23:42:12.883 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-osd (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:12.903 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-base (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:13.301 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T23:42:13.380 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T23:42:13.381 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T23:42:13.765 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mds (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:13.845 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:13.980 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:14.001 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T23:42:14.001 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T23:42:14.123 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mon (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:14.334 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T23:42:14.335 INFO:teuthology.orchestra.run.smithi167.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T23:42:14.457 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:14.691 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T23:42:14.691 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T23:42:14.924 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:15.192 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-osd (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:15.301 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:15.427 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:15.570 INFO:teuthology.orchestra.run.smithi167.stdout:Setting up ceph-volume (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:15.707 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T23:42:15.707 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T23:42:15.932 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T23:42:16.083 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T23:42:16.228 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-k8sevents (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:16.353 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:16.360 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T23:42:16.488 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mon (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:16.707 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T23:42:16.708 INFO:teuthology.orchestra.run.smithi012.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T23:42:17.156 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-cephadm (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:17.299 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:17.424 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-mgr-dashboard (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:17.793 INFO:teuthology.orchestra.run.smithi012.stdout:Setting up ceph-volume (18.2.4-845-ge4e227b3-1focal) ... 2024-09-13T23:42:18.163 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-13T23:42:18.315 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-13T23:42:18.599 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-13T23:42:19.121 INFO:teuthology.orchestra.run.smithi167.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T23:42:19.716 INFO:teuthology.orchestra.run.smithi167.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T23:42:19.722 DEBUG:teuthology.parallel:result is None 2024-09-13T23:42:21.637 INFO:teuthology.orchestra.run.smithi012.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-13T23:42:22.254 INFO:teuthology.orchestra.run.smithi012.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T23:42:22.258 DEBUG:teuthology.parallel:result is None 2024-09-13T23:42:22.259 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-13T23:42:22.425 DEBUG:teuthology.orchestra.run.smithi012:> dpkg-query -W -f '${Version}' ceph 2024-09-13T23:42:22.448 INFO:teuthology.orchestra.run.smithi012.stdout:18.2.4-845-ge4e227b3-1focal 2024-09-13T23:42:22.449 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845-ge4e227b3-1focal 2024-09-13T23:42:22.449 INFO:teuthology.task.install:The correct ceph version 18.2.4-845-ge4e227b3-1focal is installed. 2024-09-13T23:42:22.451 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=e4e227b37858f5109138494e01e5ba1ba7b8ffe8 2024-09-13T23:42:22.595 DEBUG:teuthology.orchestra.run.smithi167:> dpkg-query -W -f '${Version}' ceph 2024-09-13T23:42:22.618 INFO:teuthology.orchestra.run.smithi167.stdout:18.2.4-845-ge4e227b3-1focal 2024-09-13T23:42:22.618 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-845-ge4e227b3-1focal 2024-09-13T23:42:22.618 INFO:teuthology.task.install:The correct ceph version 18.2.4-845-ge4e227b3-1focal is installed. 2024-09-13T23:42:22.620 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T23:42:22.620 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:42:22.621 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T23:42:22.638 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:22.638 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T23:42:22.677 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T23:42:22.677 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:42:22.677 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T23:42:22.692 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T23:42:22.750 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:22.750 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T23:42:22.763 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T23:42:22.824 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T23:42:22.824 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:42:22.824 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T23:42:22.841 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T23:42:22.850 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:22.850 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T23:42:22.864 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T23:42:22.923 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T23:42:22.924 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:42:22.924 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T23:42:22.940 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T23:42:23.002 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:23.002 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T23:42:23.016 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T23:42:23.071 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T23:42:23.169 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T23:42:23.169 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 777 /var/log/ceph 2024-09-13T23:42:23.172 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /var/log/ceph 2024-09-13T23:42:23.187 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T23:42:23.187 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T23:42:23.228 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T23:42:23.243 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T23:42:23.244 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T23:42:23.246 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T23:42:23.308 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T23:42:23.308 INFO:tasks.ceph:config {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T23:42:23.308 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi/7904450', 'branch': 'reef', 'description': 'rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_20.04}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7904450', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'wait-for-scrub': False}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}}, 'rgw': {'ec-data-pool': True, 'erasure_code_profile': {'crush-failure-domain': 'osd', 'k': 3, 'm': 1, 'name': 'testprofile'}, 'frontend': 'beast'}, 's3tests': {'slow_backend': True}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'osd.0'], ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 4285, 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'sleep_before_teardown': 0, 'subset': '0/1', 'suite': 'rgw', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_e4e227b37858f5109138494e01e5ba1ba7b8ffe8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'targets': {'smithi012.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi167.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': 'e4e227b37858f5109138494e01e5ba1ba7b8ffe8', 'wait-for-scrub': False, 'cluster': 'ceph'}}, {'rgw': {'client.0': None}}, {'radosgw-admin': None}, {'radosgw-admin-rest': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_22:40:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T23:42:23.309 DEBUG:teuthology.orchestra.run.smithi012:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T23:42:23.311 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T23:42:23.318 DEBUG:teuthology.orchestra.run.smithi012:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T23:42:23.320 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T23:42:23.334 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:42:23.334 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T23:42:23.338 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T23:42:23.339 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_1 2024-09-13T23:42:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T23:42:23.390 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T23:42:23.390 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 358 Links: 1 2024-09-13T23:42:23.390 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T23:42:23.391 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-13 23:38:54.511543605 +0000 2024-09-13T23:42:23.391 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-13 23:38:54.303548389 +0000 2024-09-13T23:42:23.391 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-13 23:38:54.303548389 +0000 2024-09-13T23:42:23.391 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-09-13T23:42:23.391 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T23:42:23.404 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-13T23:42:23.404 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-13T23:42:23.405 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000428704 s, 1.2 MB/s 2024-09-13T23:42:23.406 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T23:42:23.414 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_2 2024-09-13T23:42:23.465 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T23:42:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T23:42:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 367 Links: 1 2024-09-13T23:42:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T23:42:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-13 23:38:54.947533573 +0000 2024-09-13T23:42:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-13 23:38:54.731538544 +0000 2024-09-13T23:42:23.466 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-13 23:38:54.731538544 +0000 2024-09-13T23:42:23.466 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-09-13T23:42:23.467 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T23:42:23.481 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-13T23:42:23.481 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-13T23:42:23.482 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000428504 s, 1.2 MB/s 2024-09-13T23:42:23.483 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T23:42:23.492 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_3 2024-09-13T23:42:23.541 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T23:42:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T23:42:23.542 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 374 Links: 1 2024-09-13T23:42:23.542 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T23:42:23.542 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-13 23:38:55.375523726 +0000 2024-09-13T23:42:23.542 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-13 23:38:55.159528696 +0000 2024-09-13T23:42:23.542 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-13 23:38:55.159528696 +0000 2024-09-13T23:42:23.542 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-09-13T23:42:23.542 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T23:42:23.554 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-13T23:42:23.554 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-13T23:42:23.554 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000415135 s, 1.2 MB/s 2024-09-13T23:42:23.556 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T23:42:23.565 DEBUG:teuthology.orchestra.run.smithi012:> stat /dev/vg_nvme/lv_4 2024-09-13T23:42:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T23:42:23.572 INFO:teuthology.orchestra.run.smithi012.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T23:42:23.572 INFO:teuthology.orchestra.run.smithi012.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-09-13T23:42:23.572 INFO:teuthology.orchestra.run.smithi012.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T23:42:23.572 INFO:teuthology.orchestra.run.smithi012.stdout:Access: 2024-09-13 23:38:55.839513051 +0000 2024-09-13T23:42:23.573 INFO:teuthology.orchestra.run.smithi012.stdout:Modify: 2024-09-13 23:38:55.611518297 +0000 2024-09-13T23:42:23.573 INFO:teuthology.orchestra.run.smithi012.stdout:Change: 2024-09-13 23:38:55.611518297 +0000 2024-09-13T23:42:23.573 INFO:teuthology.orchestra.run.smithi012.stdout: Birth: - 2024-09-13T23:42:23.573 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T23:42:23.629 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records in 2024-09-13T23:42:23.630 INFO:teuthology.orchestra.run.smithi012.stderr:1+0 records out 2024-09-13T23:42:23.630 INFO:teuthology.orchestra.run.smithi012.stderr:512 bytes copied, 0.000378872 s, 1.4 MB/s 2024-09-13T23:42:23.631 DEBUG:teuthology.orchestra.run.smithi012:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T23:42:23.639 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-09-13T23:42:23.639 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:23.639 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T23:42:23.647 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T23:42:23.647 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2024-09-13T23:42:23.654 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T23:42:23.654 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T23:42:23.654 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 361 Links: 1 2024-09-13T23:42:23.654 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T23:42:23.654 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-09-13 23:38:53.677829176 +0000 2024-09-13T23:42:23.654 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-09-13 23:38:53.673829268 +0000 2024-09-13T23:42:23.654 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-09-13 23:38:53.673829268 +0000 2024-09-13T23:42:23.654 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-09-13T23:42:23.655 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T23:42:23.666 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-09-13T23:42:23.667 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-09-13T23:42:23.667 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000370577 s, 1.4 MB/s 2024-09-13T23:42:23.668 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T23:42:23.674 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2024-09-13T23:42:23.723 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T23:42:23.723 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T23:42:23.723 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 365 Links: 1 2024-09-13T23:42:23.724 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T23:42:23.724 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-09-13 23:38:54.329814156 +0000 2024-09-13T23:42:23.724 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-09-13 23:38:54.125818855 +0000 2024-09-13T23:42:23.724 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-09-13 23:38:54.125818855 +0000 2024-09-13T23:42:23.724 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-09-13T23:42:23.724 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T23:42:23.736 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-09-13T23:42:23.736 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-09-13T23:42:23.736 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000331693 s, 1.5 MB/s 2024-09-13T23:42:23.737 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T23:42:23.746 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2024-09-13T23:42:23.751 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T23:42:23.752 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T23:42:23.752 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 373 Links: 1 2024-09-13T23:42:23.752 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T23:42:23.752 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-09-13 23:38:54.745804573 +0000 2024-09-13T23:42:23.752 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-09-13 23:38:54.517809826 +0000 2024-09-13T23:42:23.752 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-09-13 23:38:54.517809826 +0000 2024-09-13T23:42:23.752 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-09-13T23:42:23.752 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T23:42:23.763 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-09-13T23:42:23.763 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-09-13T23:42:23.764 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000377708 s, 1.4 MB/s 2024-09-13T23:42:23.765 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T23:42:23.817 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2024-09-13T23:42:23.824 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T23:42:23.825 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T23:42:23.825 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 378 Links: 1 2024-09-13T23:42:23.825 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T23:42:23.825 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2024-09-13 23:38:55.217793699 +0000 2024-09-13T23:42:23.825 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2024-09-13 23:38:54.973799321 +0000 2024-09-13T23:42:23.825 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2024-09-13 23:38:54.973799321 +0000 2024-09-13T23:42:23.825 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2024-09-13T23:42:23.826 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T23:42:23.838 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2024-09-13T23:42:23.838 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2024-09-13T23:42:23.838 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000241411 s, 2.1 MB/s 2024-09-13T23:42:23.839 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T23:42:23.889 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-09-13T23:42:23.889 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-09-13T23:42:23.889 INFO:tasks.ceph:Generating config... 2024-09-13T23:42:23.891 INFO:tasks.ceph:[client] debug ms = 1 2024-09-13T23:42:23.891 INFO:tasks.ceph:[client] rgw gc obj min wait = 15 2024-09-13T23:42:23.891 INFO:tasks.ceph:[client] debug rgw = 20 2024-09-13T23:42:23.891 INFO:tasks.ceph:[client] setgroup = ceph 2024-09-13T23:42:23.891 INFO:tasks.ceph:[client] setuser = ceph 2024-09-13T23:42:23.891 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T23:42:23.891 INFO:tasks.ceph:[osd] debug objclass = 20 2024-09-13T23:42:23.891 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-13T23:42:23.891 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-13T23:42:23.892 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-13T23:42:23.892 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T23:42:23.892 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T23:42:23.892 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T23:42:23.892 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T23:42:23.892 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T23:42:23.892 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T23:42:23.892 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T23:42:23.892 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T23:42:23.893 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T23:42:23.893 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T23:42:23.893 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T23:42:23.893 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T23:42:23.893 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T23:42:23.893 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T23:42:23.893 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T23:42:23.893 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T23:42:23.893 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T23:42:23.893 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T23:42:23.934 INFO:teuthology.orchestra.run.smithi012.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T23:42:23.950 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T23:42:23.991 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T23:42:24.005 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.12')] 2024-09-13T23:42:24.006 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.12'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}} 2024-09-13T23:42:24.007 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:42:24.007 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T23:42:24.012 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.12 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T23:42:24.095 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:42:24.095 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:42:24.096 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T23:42:24.096 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: generated fsid dc64727a-88fa-459b-b2f7-e6853bb706a5 2024-09-13T23:42:24.096 INFO:teuthology.orchestra.run.smithi012.stdout:setting min_mon_release = pacific 2024-09-13T23:42:24.096 INFO:teuthology.orchestra.run.smithi012.stdout:epoch 0 2024-09-13T23:42:24.096 INFO:teuthology.orchestra.run.smithi012.stdout:fsid dc64727a-88fa-459b-b2f7-e6853bb706a5 2024-09-13T23:42:24.096 INFO:teuthology.orchestra.run.smithi012.stdout:last_changed 2024-09-13T23:42:24.094128+0000 2024-09-13T23:42:24.096 INFO:teuthology.orchestra.run.smithi012.stdout:created 2024-09-13T23:42:24.094128+0000 2024-09-13T23:42:24.096 INFO:teuthology.orchestra.run.smithi012.stdout:min_mon_release 16 (pacific) 2024-09-13T23:42:24.096 INFO:teuthology.orchestra.run.smithi012.stdout:election_strategy: 1 2024-09-13T23:42:24.096 INFO:teuthology.orchestra.run.smithi012.stdout:0: [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] mon.a 2024-09-13T23:42:24.097 INFO:teuthology.orchestra.run.smithi012.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-09-13T23:42:24.098 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T23:42:24.105 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID dc64727a-88fa-459b-b2f7-e6853bb706a5... 2024-09-13T23:42:24.106 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T23:42:24.108 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T23:42:24.137 INFO:teuthology.orchestra.run.smithi012.stdout:[global] 2024-09-13T23:42:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: chdir = "" 2024-09-13T23:42:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T23:42:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: auth supported = cephx 2024-09-13T23:42:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: filestore xattr use omap = true 2024-09-13T23:42:24.137 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: mon clock drift allowed = 1.000 2024-09-13T23:42:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: osd crush chooseleaf type = 0 2024-09-13T23:42:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: auth debug = true 2024-09-13T23:42:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on old message = true 2024-09-13T23:42:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: ms die on bug = true 2024-09-13T23:42:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.138 INFO:teuthology.orchestra.run.smithi012.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: mon pg warn max object skew = 0 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default size = 2 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow primary affinity = true 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd allow pg remap = true 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on legacy crush tunables = false 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on crush straw calc version zero = false 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on no sortbitwise = false 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on osd down out interval zero = false 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: mon warn on too few osds = false 2024-09-13T23:42:24.139 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: mon_allow_pool_size_one = true 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: osd default data pool replay window = 5 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: mon allow pool delete = true 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: mon cluster log file level = debug 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: debug asserts on shutdown = true 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: mon health detail to clog = false 2024-09-13T23:42:24.140 INFO:teuthology.orchestra.run.smithi012.stdout: mon host = 172.21.15.12 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: fsid = dc64727a-88fa-459b-b2f7-e6853bb706a5 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout:[osd] 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: osd journal size = 100 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub load threshold = 5.0 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: osd scrub max interval = 600 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: osd mclock profile = high_recovery_ops 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: osd recover clone overlap = true 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: osd recovery max chunk = 1048576 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug shutdown = true 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug op order = true 2024-09-13T23:42:24.141 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug verify stray on activate = true 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: osd open classes on start = true 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug pg log writeout = true 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: osd deep scrub update digest min age = 30 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: osd map max advance = 10 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: journal zero on create = true 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: filestore ondisk finisher threads = 3 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: filestore apply finisher threads = 3 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.142 INFO:teuthology.orchestra.run.smithi012.stdout: bdev debug aio = true 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: osd debug misdirected ops = true 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: debug objclass = 20 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: bdev async discard = True 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: bdev enable discard = True 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore allocator = bitmap 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore block size = 96636764160 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: bluestore fsck on mount = True 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluefs = 1/20 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: debug bluestore = 1/20 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: debug osd = 20 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: debug rocksdb = 4/10 2024-09-13T23:42:24.143 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd full ratio = 0.9 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd nearfull ratio = 0.8 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: osd failsafe full ratio = 0.95 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: osd objectstore = bluestore 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr] 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: debug mgr = 20 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: mgr/telemetry/nag = false 2024-09-13T23:42:24.144 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout:[mon] 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: debug mon = 20 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: debug paxos = 20 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: debug auth = 20 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: mon data avail warn = 5 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: mon mgr mkfs grace = 240 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min pgs per osd = 4 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd reporter subtree level = osd 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: mon osd prime pg temp = true 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: mon reweight min bytes per osd = 10 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T23:42:24.145 INFO:teuthology.orchestra.run.smithi012.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: # 1m isn't quite enough 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: mon_down_mkfs_grace = 2m 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: mon_warn_on_filestore_osds = false 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout:[client] 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: rgw cache enabled = true 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable ops log = true 2024-09-13T23:42:24.146 INFO:teuthology.orchestra.run.smithi012.stdout: rgw enable usage log = true 2024-09-13T23:42:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T23:42:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T23:42:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: debug ms = 1 2024-09-13T23:42:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: rgw gc obj min wait = 15 2024-09-13T23:42:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: debug rgw = 20 2024-09-13T23:42:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: setgroup = ceph 2024-09-13T23:42:24.147 INFO:teuthology.orchestra.run.smithi012.stdout: setuser = ceph 2024-09-13T23:42:24.147 INFO:teuthology.orchestra.run.smithi012.stdout:[mon.a] 2024-09-13T23:42:24.148 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2024-09-13T23:42:24.148 INFO:teuthology.orchestra.run.smithi167.stdout: chdir = "" 2024-09-13T23:42:24.148 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T23:42:24.148 INFO:teuthology.orchestra.run.smithi167.stdout: auth supported = cephx 2024-09-13T23:42:24.148 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.148 INFO:teuthology.orchestra.run.smithi167.stdout: filestore xattr use omap = true 2024-09-13T23:42:24.148 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.148 INFO:teuthology.orchestra.run.smithi167.stdout: mon clock drift allowed = 1.000 2024-09-13T23:42:24.148 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: osd crush chooseleaf type = 0 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: auth debug = true 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on old message = true 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on bug = true 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: mon pg warn max object skew = 0 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default size = 2 2024-09-13T23:42:24.149 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow primary affinity = true 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow pg remap = true 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on legacy crush tunables = false 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on crush straw calc version zero = false 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on no sortbitwise = false 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on osd down out interval zero = false 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on too few osds = false 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: mon_allow_pool_size_one = true 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T23:42:24.150 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout: osd default data pool replay window = 5 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout: mon allow pool delete = true 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout: mon cluster log file level = debug 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout: debug asserts on shutdown = true 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout: mon health detail to clog = false 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout: mon host = 172.21.15.12 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = dc64727a-88fa-459b-b2f7-e6853bb706a5 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout:[osd] 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout: osd journal size = 100 2024-09-13T23:42:24.151 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub load threshold = 5.0 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub max interval = 600 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: osd mclock profile = high_recovery_ops 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: osd recover clone overlap = true 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: osd recovery max chunk = 1048576 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug shutdown = true 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug op order = true 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify stray on activate = true 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: osd open classes on start = true 2024-09-13T23:42:24.152 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug pg log writeout = true 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: osd deep scrub update digest min age = 30 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: osd map max advance = 10 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: journal zero on create = true 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: filestore ondisk finisher threads = 3 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: filestore apply finisher threads = 3 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: bdev debug aio = true 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug misdirected ops = true 2024-09-13T23:42:24.153 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: debug objclass = 20 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: bdev async discard = True 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: bdev enable discard = True 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore allocator = bitmap 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore block size = 96636764160 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore fsck on mount = True 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluefs = 1/20 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluestore = 1/20 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: debug osd = 20 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: debug rocksdb = 4/10 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd full ratio = 0.9 2024-09-13T23:42:24.154 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd nearfull ratio = 0.8 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: osd failsafe full ratio = 0.95 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: osd objectstore = bluestore 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr] 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: debug mgr = 20 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: mgr/telemetry/nag = false 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout:[mon] 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-09-13T23:42:24.155 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: debug paxos = 20 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: mon data avail warn = 5 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: mon mgr mkfs grace = 240 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd reporter subtree level = osd 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd prime pg temp = true 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T23:42:24.156 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout: # 1m isn't quite enough 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout: mon_down_mkfs_grace = 2m 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_filestore_osds = false 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout:[client] 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout: rgw cache enabled = true 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable ops log = true 2024-09-13T23:42:24.157 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable usage log = true 2024-09-13T23:42:24.158 INFO:teuthology.orchestra.run.smithi167.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T23:42:24.158 INFO:teuthology.orchestra.run.smithi167.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T23:42:24.158 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2024-09-13T23:42:24.158 INFO:teuthology.orchestra.run.smithi167.stdout: rgw gc obj min wait = 15 2024-09-13T23:42:24.158 INFO:teuthology.orchestra.run.smithi167.stdout: debug rgw = 20 2024-09-13T23:42:24.158 INFO:teuthology.orchestra.run.smithi167.stdout: setgroup = ceph 2024-09-13T23:42:24.158 INFO:teuthology.orchestra.run.smithi167.stdout: setuser = ceph 2024-09-13T23:42:24.158 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.a] 2024-09-13T23:42:24.159 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T23:42:24.160 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T23:42:24.258 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T23:42:24.259 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:42:24.259 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T23:42:24.265 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:42:24.265 DEBUG:teuthology.orchestra.run.smithi012:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T23:42:24.271 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi012.front.sepia.ceph.com 2024-09-13T23:42:24.271 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:42:24.271 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T23:42:24.271 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T23:42:24.293 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:42:24.294 DEBUG:teuthology.orchestra.run.smithi012:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T23:42:24.341 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi167.front.sepia.ceph.com 2024-09-13T23:42:24.341 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:24.341 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T23:42:24.341 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T23:42:24.368 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:24.369 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T23:42:24.374 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T23:42:24.374 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T23:42:24.374 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T23:42:24.424 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T23:42:24.433 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T23:42:24.433 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T23:42:24.433 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T23:42:24.473 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T23:42:24.485 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T23:42:24.485 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi012.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-09-13T23:42:24.486 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T23:42:24.502 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-09-13T23:42:24.502 INFO:tasks.ceph:role: osd.0 2024-09-13T23:42:24.502 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-13T23:42:24.502 DEBUG:teuthology.orchestra.run.smithi012:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T23:42:25.016 INFO:teuthology.orchestra.run.smithi012.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T23:42:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T23:42:25.016 INFO:teuthology.orchestra.run.smithi012.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T23:42:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: = reflink=1 2024-09-13T23:42:25.017 INFO:teuthology.orchestra.run.smithi012.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T23:42:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: = sunit=0 swidth=0 blks 2024-09-13T23:42:25.017 INFO:teuthology.orchestra.run.smithi012.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T23:42:25.017 INFO:teuthology.orchestra.run.smithi012.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T23:42:25.017 INFO:teuthology.orchestra.run.smithi012.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T23:42:25.017 INFO:teuthology.orchestra.run.smithi012.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T23:42:25.019 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi012.front.sepia.ceph.com -o noatime 2024-09-13T23:42:25.019 DEBUG:teuthology.orchestra.run.smithi012:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T23:42:25.108 DEBUG:teuthology.orchestra.run.smithi012:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T23:42:25.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T23:42:25.122 INFO:teuthology.orchestra.run.smithi012.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T23:42:25.122 DEBUG:teuthology.orchestra.run.smithi012:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T23:42:25.202 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:25.193+0000 7f62cc788880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T23:42:25.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:25.193+0000 7f62cc788880 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T23:42:25.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:25.197+0000 7f62cc788880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T23:42:25.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:25.197+0000 7f62cc788880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T23:42:25.203 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:25.197+0000 7f62cc788880 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T23:42:25.210 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:25.201+0000 7f62cc788880 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T23:42:28.198 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T23:42:28.213 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T23:42:28.229 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-09-13T23:42:28.230 INFO:tasks.ceph:role: osd.1 2024-09-13T23:42:28.230 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-13T23:42:28.230 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T23:42:28.754 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T23:42:28.754 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T23:42:28.755 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T23:42:28.755 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2024-09-13T23:42:28.755 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T23:42:28.755 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-09-13T23:42:28.755 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T23:42:28.755 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T23:42:28.755 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T23:42:28.755 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T23:42:28.756 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-09-13T23:42:28.756 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-09-13T23:42:28.856 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T23:42:28.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T23:42:28.867 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T23:42:28.867 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T23:42:28.925 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-09-13T23:42:28.925 INFO:tasks.ceph:role: osd.2 2024-09-13T23:42:28.925 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-13T23:42:28.926 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T23:42:29.432 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T23:42:29.432 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T23:42:29.432 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T23:42:29.432 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2024-09-13T23:42:29.432 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T23:42:29.432 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-09-13T23:42:29.432 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T23:42:29.432 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T23:42:29.432 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T23:42:29.432 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T23:42:29.434 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-09-13T23:42:29.434 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2024-09-13T23:42:29.474 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T23:42:29.484 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T23:42:29.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T23:42:29.485 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T23:42:29.497 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-09-13T23:42:29.498 INFO:tasks.ceph:role: osd.3 2024-09-13T23:42:29.498 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-13T23:42:29.498 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T23:42:29.988 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T23:42:29.989 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T23:42:29.989 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T23:42:29.989 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 2024-09-13T23:42:29.989 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T23:42:29.989 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2024-09-13T23:42:29.989 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T23:42:29.989 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T23:42:29.989 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T23:42:29.989 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T23:42:29.991 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2024-09-13T23:42:29.991 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2024-09-13T23:42:30.042 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T23:42:30.053 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T23:42:30.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T23:42:30.054 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T23:42:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:30.164+0000 7f0e3a813880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T23:42:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:30.164+0000 7f0e3a813880 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T23:42:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:30.164+0000 7f0e3a813880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T23:42:30.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:30.164+0000 7f0e3a813880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T23:42:30.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:30.164+0000 7f0e3a813880 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T23:42:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:30.172+0000 7f0e3a813880 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T23:42:33.165 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T23:42:33.182 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T23:42:33.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:33.220+0000 7f871bf41880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T23:42:33.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:33.220+0000 7f871bf41880 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T23:42:33.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:33.220+0000 7f871bf41880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T23:42:33.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:33.220+0000 7f871bf41880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T23:42:33.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:33.220+0000 7f871bf41880 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T23:42:33.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:33.224+0000 7f871bf41880 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T23:42:36.203 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T23:42:36.219 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T23:42:36.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:36.252+0000 7fd2e4834880 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T23:42:36.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:36.252+0000 7fd2e4834880 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T23:42:36.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:36.252+0000 7fd2e4834880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T23:42:36.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:36.252+0000 7fd2e4834880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T23:42:36.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:36.252+0000 7fd2e4834880 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T23:42:36.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:36.256+0000 7fd2e4834880 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T23:42:39.264 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T23:42:39.278 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T23:42:39.279 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:42:39.279 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T23:42:39.295 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:39.295 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T23:42:39.336 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:39.336 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T23:42:39.346 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:39.346 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T23:42:39.360 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:39.360 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T23:42:39.371 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:39.371 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T23:42:39.419 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T23:42:39.419 DEBUG:teuthology.orchestra.run.smithi012:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T23:42:39.435 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.0] 2024-09-13T23:42:39.435 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDhzeRmnBnwCxAAm3z8OlJ/hflJ+fBn2U5BWQ== 2024-09-13T23:42:39.435 INFO:teuthology.orchestra.run.smithi012.stdout:[mgr.x] 2024-09-13T23:42:39.435 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDgzeRmT/kbGRAAq9TiSNF51WNgDJmLrXQrhw== 2024-09-13T23:42:39.435 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.1] 2024-09-13T23:42:39.435 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDmzeRm83DyCRAAB4WB7nbUI47WyFSQYv/cFQ== 2024-09-13T23:42:39.436 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.2] 2024-09-13T23:42:39.436 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDpzeRm6LVuDRAA1Cc889h9ielF581irdof/A== 2024-09-13T23:42:39.436 INFO:teuthology.orchestra.run.smithi012.stdout:[osd.3] 2024-09-13T23:42:39.436 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDszeRm+O5cDxAAt9ZfdbzBFuU2xeuxfNiglQ== 2024-09-13T23:42:39.436 INFO:teuthology.orchestra.run.smithi012.stdout:[client.0] 2024-09-13T23:42:39.436 INFO:teuthology.orchestra.run.smithi012.stdout: key = AQDgzeRmWWUKHBAAGZ0+yxECCgXHpKTDu7bNxQ== 2024-09-13T23:42:39.437 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T23:42:39.491 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T23:42:39.540 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T23:42:39.585 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T23:42:39.632 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T23:42:39.681 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T23:42:39.733 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T23:42:39.734 DEBUG:teuthology.orchestra.run.smithi012:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T23:42:39.749 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T23:42:39.837 DEBUG:teuthology.orchestra.run.smithi012:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T23:42:39.852 DEBUG:teuthology.orchestra.run.smithi012:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T23:42:39.858 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T23:42:39.859 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T23:42:39.859 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T23:42:39.904 INFO:tasks.ceph.mon.a:Started 2024-09-13T23:42:39.905 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T23:42:39.905 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T23:42:39.905 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T23:42:39.907 INFO:tasks.ceph.mgr.x:Started 2024-09-13T23:42:39.907 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T23:42:39.908 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T23:42:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.009+0000 7fb5beda0700 1 Processor -- start 2024-09-13T23:42:40.015 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.009+0000 7fb5beda0700 1 -- start start 2024-09-13T23:42:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.013+0000 7fb5beda0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb5b806e910 0x7fb5b806ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.013+0000 7fb5beda0700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b8070050 con 0x7fb5b806f230 2024-09-13T23:42:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.013+0000 7fb5beda0700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b8070a70 con 0x7fb5b806e910 2024-09-13T23:42:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.013+0000 7fb5bcb3c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fb5b806e910 0x7fb5b806ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:40.016 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.013+0000 7fb5b7fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fb5b806f230 0x7fb5b8125c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33736/0 (socket says 172.21.15.12:33736) 2024-09-13T23:42:40.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.013+0000 7fb5b7fff700 1 -- 172.21.15.12:0/3354228644 learned_addr learned my addr 172.21.15.12:0/3354228644 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:42:40.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.013+0000 7fb5bcb3c700 1 -- 172.21.15.12:0/3354228644 >> v2:172.21.15.12:3300/0 conn(0x7fb5b806e910 msgr2=0x7fb5b806ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-13T23:42:40.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.013+0000 7fb5bcb3c700 1 -- 172.21.15.12:0/3354228644 >> v2:172.21.15.12:3300/0 conn(0x7fb5b806e910 msgr2=0x7fb5b806ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T23:42:40.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.013+0000 7fb5bcb3c700 1 --2- 172.21.15.12:0/3354228644 >> v2:172.21.15.12:3300/0 conn(0x7fb5b806e910 0x7fb5b806ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T23:42:40.017 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.013+0000 7fb5bcb3c700 1 --2- 172.21.15.12:0/3354228644 >> v2:172.21.15.12:3300/0 conn(0x7fb5b806e910 0x7fb5b806ecf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T23:42:40.065 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5b77fe700 1 -- 172.21.15.12:0/3354228644 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034473603 0 0) 0x7fb5b8070050 con 0x7fb5b806f230 2024-09-13T23:42:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5b77fe700 1 -- 172.21.15.12:0/3354228644 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5a00035d0 con 0x7fb5b806f230 2024-09-13T23:42:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5b77fe700 1 -- 172.21.15.12:0/3354228644 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1545835458 0 0) 0x7fb5a00035d0 con 0x7fb5b806f230 2024-09-13T23:42:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5b77fe700 1 -- 172.21.15.12:0/3354228644 >> v2:172.21.15.12:3300/0 conn(0x7fb5b806e910 msgr2=0x7fb5b806ecf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T23:42:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5b77fe700 1 --2- 172.21.15.12:0/3354228644 >> v2:172.21.15.12:3300/0 conn(0x7fb5b806e910 0x7fb5b806ecf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5b77fe700 1 -- 172.21.15.12:0/3354228644 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b806fe50 con 0x7fb5b806f230 2024-09-13T23:42:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5b77fe700 1 -- 172.21.15.12:0/3354228644 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 222865856 0 0) 0x7fb5a80015a0 con 0x7fb5b806f230 2024-09-13T23:42:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5b77fe700 1 -- 172.21.15.12:0/3354228644 >> v1:172.21.15.12:6789/0 conn(0x7fb5b806f230 legacy=0x7fb5b8125c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5b77fe700 1 --2- 172.21.15.12:0/3354228644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 0x7fb5a0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5b77fe700 1 -- 172.21.15.12:0/3354228644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a00045c0 con 0x7fb5a0003a50 2024-09-13T23:42:40.066 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5bcb3c700 1 --2- 172.21.15.12:0/3354228644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 0x7fb5a0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5bcb3c700 1 -- 172.21.15.12:0/3354228644 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a00035d0 con 0x7fb5a0003a50 2024-09-13T23:42:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.061+0000 7fb5bcb3c700 1 --2- 172.21.15.12:0/3354228644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 0x7fb5a0003e50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0007880 tx=0x7fb5a00058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a046fcd8aa6335 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.065+0000 7fb5b77fe700 1 -- 172.21.15.12:0/3354228644 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5b0004790 con 0x7fb5a0003a50 2024-09-13T23:42:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.065+0000 7fb5b77fe700 1 -- 172.21.15.12:0/3354228644 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b0011040 con 0x7fb5a0003a50 2024-09-13T23:42:40.067 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.065+0000 7fb5b77fe700 1 -- 172.21.15.12:0/3354228644 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5b0007ac0 con 0x7fb5a0003a50 2024-09-13T23:42:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.065+0000 7fb5beda0700 1 -- 172.21.15.12:0/3354228644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 msgr2=0x7fb5a0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.065+0000 7fb5beda0700 1 --2- 172.21.15.12:0/3354228644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 0x7fb5a0003e50 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fb5b0007880 tx=0x7fb5a00058a0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.065+0000 7fb5beda0700 1 -- 172.21.15.12:0/3354228644 shutdown_connections 2024-09-13T23:42:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.065+0000 7fb5beda0700 1 --2- 172.21.15.12:0/3354228644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 0x7fb5a0003e50 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.065+0000 7fb5beda0700 1 --2- 172.21.15.12:0/3354228644 >> v2:172.21.15.12:3300/0 conn(0x7fb5b806e910 0x7fb5b806ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.065+0000 7fb5beda0700 1 -- 172.21.15.12:0/3354228644 >> 172.21.15.12:0/3354228644 conn(0x7fb5b8097a10 msgr2=0x7fb5b8097df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.065+0000 7fb5beda0700 1 -- 172.21.15.12:0/3354228644 shutdown_connections 2024-09-13T23:42:40.069 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.065+0000 7fb5beda0700 1 -- 172.21.15.12:0/3354228644 wait complete. 2024-09-13T23:42:40.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.069+0000 7fb5beda0700 1 Processor -- start 2024-09-13T23:42:40.071 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.069+0000 7fb5beda0700 1 -- start start 2024-09-13T23:42:40.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.069+0000 7fb5beda0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 0x7fb5b81bd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:40.073 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.069+0000 7fb5beda0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b811cdb0 con 0x7fb5a0003a50 2024-09-13T23:42:40.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.069+0000 7fb5bcb3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 0x7fb5b81bd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:40.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.069+0000 7fb5bcb3c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 0x7fb5b81bd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44812/0 (socket says 172.21.15.12:44812) 2024-09-13T23:42:40.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.069+0000 7fb5bcb3c700 1 -- 172.21.15.12:0/2537418327 learned_addr learned my addr 172.21.15.12:0/2537418327 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:42:40.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.069+0000 7fb5bcb3c700 1 -- 172.21.15.12:0/2537418327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b8071e10 con 0x7fb5a0003a50 2024-09-13T23:42:40.074 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.069+0000 7fb5bcb3c700 1 --2- 172.21.15.12:0/2537418327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 0x7fb5b81bd3c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80668c0 tx=0x7fb5b0004b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.069+0000 7fb5b57fa700 1 -- 172.21.15.12:0/2537418327 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5b0007c70 con 0x7fb5a0003a50 2024-09-13T23:42:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.073+0000 7fb5beda0700 1 -- 172.21.15.12:0/2537418327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b811b8d0 con 0x7fb5a0003a50 2024-09-13T23:42:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.073+0000 7fb5beda0700 1 -- 172.21.15.12:0/2537418327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b811d580 con 0x7fb5a0003a50 2024-09-13T23:42:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.073+0000 7fb5b57fa700 1 -- 172.21.15.12:0/2537418327 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b0007dd0 con 0x7fb5a0003a50 2024-09-13T23:42:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.073+0000 7fb5b57fa700 1 -- 172.21.15.12:0/2537418327 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5b0028530 con 0x7fb5a0003a50 2024-09-13T23:42:40.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.073+0000 7fb5b57fa700 1 -- 172.21.15.12:0/2537418327 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7fb5b001c070 con 0x7fb5a0003a50 2024-09-13T23:42:40.076 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.073+0000 7fb5b57fa700 1 -- 172.21.15.12:0/2537418327 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb5b0015610 con 0x7fb5a0003a50 2024-09-13T23:42:40.077 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.073+0000 7fb5beda0700 1 -- 172.21.15.12:0/2537418327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b811c930 con 0x7fb5a0003a50 2024-09-13T23:42:40.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.077+0000 7fb5b57fa700 1 -- 172.21.15.12:0/2537418327 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7fb5b001a070 con 0x7fb5a0003a50 2024-09-13T23:42:40.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.149+0000 7fb5beda0700 1 -- 172.21.15.12:0/2537418327 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fb5b806be70 con 0x7fb5a0003a50 2024-09-13T23:42:40.159 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.157+0000 7fb5b57fa700 1 -- 172.21.15.12:0/2537418327 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fb5b0007500 con 0x7fb5a0003a50 2024-09-13T23:42:40.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.157+0000 7fb5beda0700 1 -- 172.21.15.12:0/2537418327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 msgr2=0x7fb5b81bd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:40.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.157+0000 7fb5beda0700 1 --2- 172.21.15.12:0/2537418327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 0x7fb5b81bd3c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb5b80668c0 tx=0x7fb5b0004b60 comp rx=0 tx=0).stop 2024-09-13T23:42:40.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.157+0000 7fb5beda0700 1 -- 172.21.15.12:0/2537418327 shutdown_connections 2024-09-13T23:42:40.160 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.157+0000 7fb5beda0700 1 --2- 172.21.15.12:0/2537418327 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fb5a0003a50 0x7fb5b81bd3c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.157+0000 7fb5beda0700 1 -- 172.21.15.12:0/2537418327 >> 172.21.15.12:0/2537418327 conn(0x7fb5b8097a10 msgr2=0x7fb5b81288f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:40.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.157+0000 7fb5beda0700 1 -- 172.21.15.12:0/2537418327 shutdown_connections 2024-09-13T23:42:40.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:40.157+0000 7fb5beda0700 1 -- 172.21.15.12:0/2537418327 wait complete. 2024-09-13T23:42:40.161 INFO:teuthology.orchestra.run.smithi012.stderr:adjusted tunables profile to default 2024-09-13T23:42:40.173 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T23:42:40.173 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T23:42:40.173 DEBUG:teuthology.orchestra.run.smithi012:> set -ex 2024-09-13T23:42:40.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T23:42:40.191 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:40.191 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T23:42:40.207 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:40.207 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T23:42:40.266 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2024-09-13T23:42:40.266 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T23:42:40.266 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:40.256+0000 7f3b562152c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T23:42:40.276 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new e2f894d3-5bb4-46dd-9be1-bbae3cddb090 0 2024-09-13T23:42:40.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.380+0000 7ff602e78700 1 Processor -- start 2024-09-13T23:42:40.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.380+0000 7ff602e78700 1 -- start start 2024-09-13T23:42:40.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.380+0000 7ff602e78700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff5fc117410 0x7ff5fc119800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:40.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.380+0000 7ff602e78700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5fc119dd0 con 0x7ff5fc114650 2024-09-13T23:42:40.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.380+0000 7ff602e78700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5fc11af70 con 0x7ff5fc117410 2024-09-13T23:42:40.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.380+0000 7ff5fbfff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff5fc117410 0x7ff5fc119800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:40.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.380+0000 7ff5fbfff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7ff5fc117410 0x7ff5fc119800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:45202/0 (socket says 172.21.15.167:45202) 2024-09-13T23:42:40.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.380+0000 7ff5fbfff700 1 -- 172.21.15.167:0/291821761 learned_addr learned my addr 172.21.15.167:0/291821761 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:42:40.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.380+0000 7ff600c14700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7ff5fc114650 0x7ff5fc114a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:57822/0 (socket says 172.21.15.167:57822) 2024-09-13T23:42:40.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.380+0000 7ff5fbfff700 1 -- 172.21.15.167:0/291821761 >> v1:172.21.15.12:6789/0 conn(0x7ff5fc114650 legacy=0x7ff5fc114a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:40.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.380+0000 7ff5fbfff700 1 -- 172.21.15.167:0/291821761 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc11b0b0 con 0x7ff5fc117410 2024-09-13T23:42:40.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.380+0000 7ff5fbfff700 1 --2- 172.21.15.167:0/291821761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5fc117410 0x7ff5fc119800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009890 tx=0x7ff5e8009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebe6bc5b4328ef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:40.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff5fb7fe700 1 -- 172.21.15.167:0/291821761 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5e8004160 con 0x7ff5fc117410 2024-09-13T23:42:40.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff5fb7fe700 1 -- 172.21.15.167:0/291821761 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e80042c0 con 0x7ff5fc117410 2024-09-13T23:42:40.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff5fb7fe700 1 -- 172.21.15.167:0/291821761 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5e8004580 con 0x7ff5fc117410 2024-09-13T23:42:40.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 -- 172.21.15.167:0/291821761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5fc117410 msgr2=0x7ff5fc119800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:40.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 --2- 172.21.15.167:0/291821761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5fc117410 0x7ff5fc119800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff5e8009890 tx=0x7ff5e8009ba0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 -- 172.21.15.167:0/291821761 shutdown_connections 2024-09-13T23:42:40.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 --2- 172.21.15.167:0/291821761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5fc117410 0x7ff5fc119800 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 -- 172.21.15.167:0/291821761 >> 172.21.15.167:0/291821761 conn(0x7ff5fc06e7b0 msgr2=0x7ff5fc070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:40.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 -- 172.21.15.167:0/291821761 shutdown_connections 2024-09-13T23:42:40.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 -- 172.21.15.167:0/291821761 wait complete. 2024-09-13T23:42:40.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 Processor -- start 2024-09-13T23:42:40.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 -- start start 2024-09-13T23:42:40.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5fc114650 0x7ff5fc1a8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:40.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc1a92b0 con 0x7ff5fc114650 2024-09-13T23:42:40.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff600c14700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5fc114650 0x7ff5fc1a8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:40.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff600c14700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5fc114650 0x7ff5fc1a8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:45206/0 (socket says 172.21.15.167:45206) 2024-09-13T23:42:40.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff600c14700 1 -- 172.21.15.167:0/499754542 learned_addr learned my addr 172.21.15.167:0/499754542 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:42:40.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff600c14700 1 -- 172.21.15.167:0/499754542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8009570 con 0x7ff5fc114650 2024-09-13T23:42:40.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff600c14700 1 --2- 172.21.15.167:0/499754542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5fc114650 0x7ff5fc1a8ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c480 tx=0x7ff5f000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:40.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff5f97fa700 1 -- 172.21.15.167:0/499754542 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5f0005c00 con 0x7ff5fc114650 2024-09-13T23:42:40.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 -- 172.21.15.167:0/499754542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5fc1a94b0 con 0x7ff5fc114650 2024-09-13T23:42:40.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff5f97fa700 1 -- 172.21.15.167:0/499754542 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f0005d60 con 0x7ff5fc114650 2024-09-13T23:42:40.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff602e78700 1 -- 172.21.15.167:0/499754542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5fc1a2d00 con 0x7ff5fc114650 2024-09-13T23:42:40.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff5f97fa700 1 -- 172.21.15.167:0/499754542 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5f00070d0 con 0x7ff5fc114650 2024-09-13T23:42:40.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff5f97fa700 1 -- 172.21.15.167:0/499754542 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7ff5f0007600 con 0x7ff5fc114650 2024-09-13T23:42:40.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.384+0000 7ff5f97fa700 1 -- 172.21.15.167:0/499754542 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff5f0012740 con 0x7ff5fc114650 2024-09-13T23:42:40.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.388+0000 7ff602e78700 1 -- 172.21.15.167:0/499754542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5fc11a9c0 con 0x7ff5fc114650 2024-09-13T23:42:40.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.392+0000 7ff5f97fa700 1 -- 172.21.15.167:0/499754542 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7ff5f0021060 con 0x7ff5fc114650 2024-09-13T23:42:40.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.476+0000 7ff602e78700 1 -- 172.21.15.167:0/499754542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e2f894d3-5bb4-46dd-9be1-bbae3cddb090", "id": 0} v 0) v1 -- 0x7ff5fc117410 con 0x7ff5fc114650 2024-09-13T23:42:40.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.480+0000 7ff5f97fa700 1 -- 172.21.15.167:0/499754542 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e2f894d3-5bb4-46dd-9be1-bbae3cddb090", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff5f0017070 con 0x7ff5fc114650 2024-09-13T23:42:40.489 INFO:teuthology.orchestra.run.smithi167.stdout:0 2024-09-13T23:42:40.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.484+0000 7ff602e78700 1 -- 172.21.15.167:0/499754542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5fc114650 msgr2=0x7ff5fc1a8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:40.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.484+0000 7ff602e78700 1 --2- 172.21.15.167:0/499754542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5fc114650 0x7ff5fc1a8ce0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c480 tx=0x7ff5f000c840 comp rx=0 tx=0).stop 2024-09-13T23:42:40.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.484+0000 7ff602e78700 1 -- 172.21.15.167:0/499754542 shutdown_connections 2024-09-13T23:42:40.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.484+0000 7ff602e78700 1 --2- 172.21.15.167:0/499754542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7ff5fc114650 0x7ff5fc1a8ce0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.484+0000 7ff602e78700 1 -- 172.21.15.167:0/499754542 >> 172.21.15.167:0/499754542 conn(0x7ff5fc06e7b0 msgr2=0x7ff5fc1177f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:40.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.484+0000 7ff602e78700 1 -- 172.21.15.167:0/499754542 shutdown_connections 2024-09-13T23:42:40.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.484+0000 7ff602e78700 1 -- 172.21.15.167:0/499754542 wait complete. 2024-09-13T23:42:40.502 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 840a539a-6bbb-4ccd-a32c-7bdf4e7e339c 1 2024-09-13T23:42:40.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.604+0000 7f013b214700 1 Processor -- start 2024-09-13T23:42:40.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.604+0000 7f013b214700 1 -- start start 2024-09-13T23:42:40.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.604+0000 7f013b214700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f013406e010 0x7f013406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:40.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.604+0000 7f013b214700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0134120400 con 0x7f013406ea20 2024-09-13T23:42:40.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.604+0000 7f013b214700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01341215a0 con 0x7f013406e010 2024-09-13T23:42:40.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.604+0000 7f0138fb0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f013406e010 0x7f013406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:40.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.604+0000 7f0138fb0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f013406e010 0x7f013406e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:45216/0 (socket says 172.21.15.167:45216) 2024-09-13T23:42:40.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.604+0000 7f0138fb0700 1 -- 172.21.15.167:0/1046166084 learned_addr learned my addr 172.21.15.167:0/1046166084 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:42:40.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.604+0000 7f0138fb0700 1 -- 172.21.15.167:0/1046166084 >> v1:172.21.15.12:6789/0 conn(0x7f013406ea20 legacy=0x7f013411ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:40.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.604+0000 7f0138fb0700 1 -- 172.21.15.167:0/1046166084 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01341216e0 con 0x7f013406e010 2024-09-13T23:42:40.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.604+0000 7f0138fb0700 1 --2- 172.21.15.167:0/1046166084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f013406e010 0x7f013406e3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0128009890 tx=0x7f0128009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9d67b671951c81 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:40.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f01337fe700 1 -- 172.21.15.167:0/1046166084 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0128004160 con 0x7f013406e010 2024-09-13T23:42:40.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f01337fe700 1 -- 172.21.15.167:0/1046166084 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01280042c0 con 0x7f013406e010 2024-09-13T23:42:40.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f01337fe700 1 -- 172.21.15.167:0/1046166084 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0128004580 con 0x7f013406e010 2024-09-13T23:42:40.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f013b214700 1 -- 172.21.15.167:0/1046166084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f013406e010 msgr2=0x7f013406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:40.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f013b214700 1 --2- 172.21.15.167:0/1046166084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f013406e010 0x7f013406e3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0128009890 tx=0x7f0128009ba0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f013b214700 1 -- 172.21.15.167:0/1046166084 shutdown_connections 2024-09-13T23:42:40.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f013b214700 1 --2- 172.21.15.167:0/1046166084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f013406e010 0x7f013406e3f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f013b214700 1 -- 172.21.15.167:0/1046166084 >> 172.21.15.167:0/1046166084 conn(0x7f013406d700 msgr2=0x7f013410ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:40.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f013b214700 1 -- 172.21.15.167:0/1046166084 shutdown_connections 2024-09-13T23:42:40.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f013b214700 1 -- 172.21.15.167:0/1046166084 wait complete. 2024-09-13T23:42:40.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f013b214700 1 Processor -- start 2024-09-13T23:42:40.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f013b214700 1 -- start start 2024-09-13T23:42:40.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f013b214700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f013406e010 0x7f01341af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:40.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f013b214700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01341af7e0 con 0x7f013406e010 2024-09-13T23:42:40.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f0138fb0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f013406e010 0x7f01341af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:40.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f0138fb0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f013406e010 0x7f01341af210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:45226/0 (socket says 172.21.15.167:45226) 2024-09-13T23:42:40.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f0138fb0700 1 -- 172.21.15.167:0/3085298924 learned_addr learned my addr 172.21.15.167:0/3085298924 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:42:40.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.608+0000 7f0138fb0700 1 -- 172.21.15.167:0/3085298924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0128009570 con 0x7f013406e010 2024-09-13T23:42:40.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.612+0000 7f0138fb0700 1 --2- 172.21.15.167:0/3085298924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f013406e010 0x7f01341af210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0128009be0 tx=0x7f0128004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:40.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.612+0000 7f01317fa700 1 -- 172.21.15.167:0/3085298924 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f012800b670 con 0x7f013406e010 2024-09-13T23:42:40.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.612+0000 7f01317fa700 1 -- 172.21.15.167:0/3085298924 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012800b7d0 con 0x7f013406e010 2024-09-13T23:42:40.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.612+0000 7f013b214700 1 -- 172.21.15.167:0/3085298924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01341af980 con 0x7f013406e010 2024-09-13T23:42:40.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.612+0000 7f01317fa700 1 -- 172.21.15.167:0/3085298924 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f012800ba70 con 0x7f013406e010 2024-09-13T23:42:40.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.612+0000 7f013b214700 1 -- 172.21.15.167:0/3085298924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01341a9230 con 0x7f013406e010 2024-09-13T23:42:40.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.612+0000 7f01317fa700 1 -- 172.21.15.167:0/3085298924 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0128021070 con 0x7f013406e010 2024-09-13T23:42:40.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.612+0000 7f01317fa700 1 -- 172.21.15.167:0/3085298924 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f012801a810 con 0x7f013406e010 2024-09-13T23:42:40.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.612+0000 7f013b214700 1 -- 172.21.15.167:0/3085298924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013406e470 con 0x7f013406e010 2024-09-13T23:42:40.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.616+0000 7f01317fa700 1 -- 172.21.15.167:0/3085298924 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f0128024d20 con 0x7f013406e010 2024-09-13T23:42:40.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.692+0000 7f013b214700 1 -- 172.21.15.167:0/3085298924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd new", "uuid": "840a539a-6bbb-4ccd-a32c-7bdf4e7e339c", "id": 1} v 0) v1 -- 0x7f0134120890 con 0x7f013406e010 2024-09-13T23:42:40.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.700+0000 7f01317fa700 1 -- 172.21.15.167:0/3085298924 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "840a539a-6bbb-4ccd-a32c-7bdf4e7e339c", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f012801aa70 con 0x7f013406e010 2024-09-13T23:42:40.708 INFO:teuthology.orchestra.run.smithi167.stdout:1 2024-09-13T23:42:40.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.700+0000 7f013b214700 1 -- 172.21.15.167:0/3085298924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f013406e010 msgr2=0x7f01341af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:40.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.700+0000 7f013b214700 1 --2- 172.21.15.167:0/3085298924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f013406e010 0x7f01341af210 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0128009be0 tx=0x7f0128004770 comp rx=0 tx=0).stop 2024-09-13T23:42:40.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.700+0000 7f013b214700 1 -- 172.21.15.167:0/3085298924 shutdown_connections 2024-09-13T23:42:40.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.700+0000 7f013b214700 1 --2- 172.21.15.167:0/3085298924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f013406e010 0x7f01341af210 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.700+0000 7f013b214700 1 -- 172.21.15.167:0/3085298924 >> 172.21.15.167:0/3085298924 conn(0x7f013406d700 msgr2=0x7f013411bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:40.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.700+0000 7f013b214700 1 -- 172.21.15.167:0/3085298924 shutdown_connections 2024-09-13T23:42:40.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.700+0000 7f013b214700 1 -- 172.21.15.167:0/3085298924 wait complete. 2024-09-13T23:42:40.722 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new b370fb03-da40-4119-8bc0-6caff41336a7 2 2024-09-13T23:42:40.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.824+0000 7efc4f4d3700 1 Processor -- start 2024-09-13T23:42:40.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.824+0000 7efc4f4d3700 1 -- start start 2024-09-13T23:42:40.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.824+0000 7efc4f4d3700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efc481146a0 0x7efc48114a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:40.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.824+0000 7efc4f4d3700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc48115140 con 0x7efc4810cd60 2024-09-13T23:42:40.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.824+0000 7efc4f4d3700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4810d920 con 0x7efc481146a0 2024-09-13T23:42:40.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.824+0000 7efc4d26f700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efc481146a0 0x7efc48114a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:40.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.824+0000 7efc4ca6e700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efc4810cd60 0x7efc4810d210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:57852/0 (socket says 172.21.15.167:57852) 2024-09-13T23:42:40.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.824+0000 7efc4ca6e700 1 -- 172.21.15.167:0/2153188437 learned_addr learned my addr 172.21.15.167:0/2153188437 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:42:40.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.824+0000 7efc4d26f700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efc481146a0 0x7efc48114a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:45242/0 (socket says 172.21.15.167:45242) 2024-09-13T23:42:40.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.824+0000 7efc4d26f700 1 -- 172.21.15.167:0/2153188437 >> v1:172.21.15.12:6789/0 conn(0x7efc4810cd60 legacy=0x7efc4810d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:40.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.824+0000 7efc4d26f700 1 -- 172.21.15.167:0/2153188437 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4810da90 con 0x7efc481146a0 2024-09-13T23:42:40.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.824+0000 7efc4d26f700 1 --2- 172.21.15.167:0/2153188437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc481146a0 0x7efc48114a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efc440097d0 tx=0x7efc44009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebaf1914ba80c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:40.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc3ffff700 1 -- 172.21.15.167:0/2153188437 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc440041c0 con 0x7efc481146a0 2024-09-13T23:42:40.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc3ffff700 1 -- 172.21.15.167:0/2153188437 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc44004320 con 0x7efc481146a0 2024-09-13T23:42:40.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc3ffff700 1 -- 172.21.15.167:0/2153188437 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc440036c0 con 0x7efc481146a0 2024-09-13T23:42:40.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2153188437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc481146a0 msgr2=0x7efc48114a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:40.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4f4d3700 1 --2- 172.21.15.167:0/2153188437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc481146a0 0x7efc48114a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efc440097d0 tx=0x7efc44009ae0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2153188437 shutdown_connections 2024-09-13T23:42:40.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4f4d3700 1 --2- 172.21.15.167:0/2153188437 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc481146a0 0x7efc48114a80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2153188437 >> 172.21.15.167:0/2153188437 conn(0x7efc4806e7b0 msgr2=0x7efc48070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:40.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2153188437 shutdown_connections 2024-09-13T23:42:40.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2153188437 wait complete. 2024-09-13T23:42:40.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4f4d3700 1 Processor -- start 2024-09-13T23:42:40.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4f4d3700 1 -- start start 2024-09-13T23:42:40.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4f4d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4810cd60 0x7efc481a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:40.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4f4d3700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc481a91e0 con 0x7efc4810cd60 2024-09-13T23:42:40.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4d26f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4810cd60 0x7efc481a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:40.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4d26f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4810cd60 0x7efc481a8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:45248/0 (socket says 172.21.15.167:45248) 2024-09-13T23:42:40.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4d26f700 1 -- 172.21.15.167:0/2951386554 learned_addr learned my addr 172.21.15.167:0/2951386554 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:42:40.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.828+0000 7efc4d26f700 1 -- 172.21.15.167:0/2951386554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc440094b0 con 0x7efc4810cd60 2024-09-13T23:42:40.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.832+0000 7efc4d26f700 1 --2- 172.21.15.167:0/2951386554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4810cd60 0x7efc481a8c10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efc44009b20 tx=0x7efc44004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:40.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.832+0000 7efc3dffb700 1 -- 172.21.15.167:0/2951386554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc44006ea0 con 0x7efc4810cd60 2024-09-13T23:42:40.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.832+0000 7efc3dffb700 1 -- 172.21.15.167:0/2951386554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc44011630 con 0x7efc4810cd60 2024-09-13T23:42:40.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.832+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2951386554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc481a9380 con 0x7efc4810cd60 2024-09-13T23:42:40.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.832+0000 7efc3dffb700 1 -- 172.21.15.167:0/2951386554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc440118d0 con 0x7efc4810cd60 2024-09-13T23:42:40.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.832+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2951386554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc481a2c30 con 0x7efc4810cd60 2024-09-13T23:42:40.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.832+0000 7efc3dffb700 1 -- 172.21.15.167:0/2951386554 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7efc44018050 con 0x7efc4810cd60 2024-09-13T23:42:40.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.832+0000 7efc3dffb700 1 -- 172.21.15.167:0/2951386554 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7efc44026d00 con 0x7efc4810cd60 2024-09-13T23:42:40.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.832+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2951386554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4811a630 con 0x7efc4810cd60 2024-09-13T23:42:40.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.836+0000 7efc3dffb700 1 -- 172.21.15.167:0/2951386554 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7efc44045e30 con 0x7efc4810cd60 2024-09-13T23:42:40.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.916+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2951386554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b370fb03-da40-4119-8bc0-6caff41336a7", "id": 2} v 0) v1 -- 0x7efc4811a7c0 con 0x7efc4810cd60 2024-09-13T23:42:40.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.924+0000 7efc3dffb700 1 -- 172.21.15.167:0/2951386554 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b370fb03-da40-4119-8bc0-6caff41336a7", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7efc44034030 con 0x7efc4810cd60 2024-09-13T23:42:40.932 INFO:teuthology.orchestra.run.smithi167.stdout:2 2024-09-13T23:42:40.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.924+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2951386554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4810cd60 msgr2=0x7efc481a8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:40.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.924+0000 7efc4f4d3700 1 --2- 172.21.15.167:0/2951386554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4810cd60 0x7efc481a8c10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efc44009b20 tx=0x7efc44004770 comp rx=0 tx=0).stop 2024-09-13T23:42:40.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.924+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2951386554 shutdown_connections 2024-09-13T23:42:40.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.928+0000 7efc4f4d3700 1 --2- 172.21.15.167:0/2951386554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efc4810cd60 0x7efc481a8c10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:40.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.928+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2951386554 >> 172.21.15.167:0/2951386554 conn(0x7efc4806e7b0 msgr2=0x7efc48119490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:40.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.928+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2951386554 shutdown_connections 2024-09-13T23:42:40.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:40.928+0000 7efc4f4d3700 1 -- 172.21.15.167:0/2951386554 wait complete. 2024-09-13T23:42:40.946 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new cf2a8b98-93cb-4290-a41d-c833ba4c647c 3 2024-09-13T23:42:41.035 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:41.028+0000 7f3b562152c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T23:42:41.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.048+0000 7effd9aab700 1 Processor -- start 2024-09-13T23:42:41.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.048+0000 7effd9aab700 1 -- start start 2024-09-13T23:42:41.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.048+0000 7effd9aab700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7effd41166a0 0x7effd4116a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:41.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.048+0000 7effd9aab700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effd4117140 con 0x7effd4110490 2024-09-13T23:42:41.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.048+0000 7effd9aab700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effd4111030 con 0x7effd41166a0 2024-09-13T23:42:41.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.048+0000 7effd8aa9700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7effd41166a0 0x7effd4116a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:41.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.048+0000 7effd3fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7effd4110490 0x7effd4110920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:57856/0 (socket says 172.21.15.167:57856) 2024-09-13T23:42:41.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.048+0000 7effd3fff700 1 -- 172.21.15.167:0/1180309605 learned_addr learned my addr 172.21.15.167:0/1180309605 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:42:41.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd8aa9700 1 -- 172.21.15.167:0/1180309605 >> v1:172.21.15.12:6789/0 conn(0x7effd4110490 legacy=0x7effd4110920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:41.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd8aa9700 1 -- 172.21.15.167:0/1180309605 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd41111a0 con 0x7effd41166a0 2024-09-13T23:42:41.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd8aa9700 1 --2- 172.21.15.167:0/1180309605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd41166a0 0x7effd4116a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7effc00097d0 tx=0x7effc0009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf47327a1998e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:41.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd37fe700 1 -- 172.21.15.167:0/1180309605 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effc00041c0 con 0x7effd41166a0 2024-09-13T23:42:41.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd37fe700 1 -- 172.21.15.167:0/1180309605 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc0004320 con 0x7effd41166a0 2024-09-13T23:42:41.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd37fe700 1 -- 172.21.15.167:0/1180309605 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effc00036c0 con 0x7effd41166a0 2024-09-13T23:42:41.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd9aab700 1 -- 172.21.15.167:0/1180309605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd41166a0 msgr2=0x7effd4116a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:41.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd9aab700 1 --2- 172.21.15.167:0/1180309605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd41166a0 0x7effd4116a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7effc00097d0 tx=0x7effc0009ae0 comp rx=0 tx=0).stop 2024-09-13T23:42:41.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd9aab700 1 -- 172.21.15.167:0/1180309605 shutdown_connections 2024-09-13T23:42:41.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd9aab700 1 --2- 172.21.15.167:0/1180309605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd41166a0 0x7effd4116a80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:41.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd9aab700 1 -- 172.21.15.167:0/1180309605 >> 172.21.15.167:0/1180309605 conn(0x7effd406d4f0 msgr2=0x7effd410ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:41.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd9aab700 1 -- 172.21.15.167:0/1180309605 shutdown_connections 2024-09-13T23:42:41.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd9aab700 1 -- 172.21.15.167:0/1180309605 wait complete. 2024-09-13T23:42:41.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd9aab700 1 Processor -- start 2024-09-13T23:42:41.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd9aab700 1 -- start start 2024-09-13T23:42:41.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd9aab700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd4110490 0x7effd41aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:41.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd9aab700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd41ab300 con 0x7effd4110490 2024-09-13T23:42:41.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd8aa9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd4110490 0x7effd41aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:41.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd8aa9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd4110490 0x7effd41aad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:45262/0 (socket says 172.21.15.167:45262) 2024-09-13T23:42:41.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.052+0000 7effd8aa9700 1 -- 172.21.15.167:0/1760644933 learned_addr learned my addr 172.21.15.167:0/1760644933 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:42:41.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.056+0000 7effd8aa9700 1 -- 172.21.15.167:0/1760644933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc00094b0 con 0x7effd4110490 2024-09-13T23:42:41.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.056+0000 7effd8aa9700 1 --2- 172.21.15.167:0/1760644933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd4110490 0x7effd41aad30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7effc00090b0 tx=0x7effc0004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:41.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.056+0000 7effd17fa700 1 -- 172.21.15.167:0/1760644933 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effc0003e70 con 0x7effd4110490 2024-09-13T23:42:41.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.056+0000 7effd17fa700 1 -- 172.21.15.167:0/1760644933 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc0011630 con 0x7effd4110490 2024-09-13T23:42:41.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.056+0000 7effd9aab700 1 -- 172.21.15.167:0/1760644933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effd41ab4a0 con 0x7effd4110490 2024-09-13T23:42:41.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.056+0000 7effd17fa700 1 -- 172.21.15.167:0/1760644933 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effc0011930 con 0x7effd4110490 2024-09-13T23:42:41.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.056+0000 7effd9aab700 1 -- 172.21.15.167:0/1760644933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effd41a4d50 con 0x7effd4110490 2024-09-13T23:42:41.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.056+0000 7effd17fa700 1 -- 172.21.15.167:0/1760644933 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7effc0018040 con 0x7effd4110490 2024-09-13T23:42:41.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.056+0000 7effd17fa700 1 -- 172.21.15.167:0/1760644933 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7effc001cd20 con 0x7effd4110490 2024-09-13T23:42:41.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.056+0000 7effd9aab700 1 -- 172.21.15.167:0/1760644933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effd411c5f0 con 0x7effd4110490 2024-09-13T23:42:41.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.068+0000 7effd17fa700 1 -- 172.21.15.167:0/1760644933 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7effc0022070 con 0x7effd4110490 2024-09-13T23:42:41.094 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:41.088+0000 7f3b562152c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T23:42:41.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.144+0000 7effd9aab700 1 -- 172.21.15.167:0/1760644933 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cf2a8b98-93cb-4290-a41d-c833ba4c647c", "id": 3} v 0) v1 -- 0x7effd411c780 con 0x7effd4110490 2024-09-13T23:42:41.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.152+0000 7effd17fa700 1 -- 172.21.15.167:0/1760644933 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cf2a8b98-93cb-4290-a41d-c833ba4c647c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7effc0016070 con 0x7effd4110490 2024-09-13T23:42:41.161 INFO:teuthology.orchestra.run.smithi167.stdout:3 2024-09-13T23:42:41.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.156+0000 7effd9aab700 1 -- 172.21.15.167:0/1760644933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd4110490 msgr2=0x7effd41aad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:41.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.156+0000 7effd9aab700 1 --2- 172.21.15.167:0/1760644933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd4110490 0x7effd41aad30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7effc00090b0 tx=0x7effc0004770 comp rx=0 tx=0).stop 2024-09-13T23:42:41.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.156+0000 7effd9aab700 1 -- 172.21.15.167:0/1760644933 shutdown_connections 2024-09-13T23:42:41.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.156+0000 7effd9aab700 1 --2- 172.21.15.167:0/1760644933 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7effd4110490 0x7effd41aad30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:41.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.156+0000 7effd9aab700 1 -- 172.21.15.167:0/1760644933 >> 172.21.15.167:0/1760644933 conn(0x7effd406d4f0 msgr2=0x7effd411b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:41.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.156+0000 7effd9aab700 1 -- 172.21.15.167:0/1760644933 shutdown_connections 2024-09-13T23:42:41.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:42:41.156+0000 7effd9aab700 1 -- 172.21.15.167:0/1760644933 wait complete. 2024-09-13T23:42:41.176 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T23:42:41.176 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T23:42:41.179 INFO:tasks.ceph.osd.0:Started 2024-09-13T23:42:41.179 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T23:42:41.179 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T23:42:41.181 INFO:tasks.ceph.osd.1:Started 2024-09-13T23:42:41.181 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T23:42:41.181 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T23:42:41.183 INFO:tasks.ceph.osd.2:Started 2024-09-13T23:42:41.183 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T23:42:41.183 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T23:42:41.185 INFO:tasks.ceph.osd.3:Started 2024-09-13T23:42:41.186 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T23:42:41.288 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:41.280+0000 7f3b562152c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T23:42:41.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e74197700 1 Processor -- start 2024-09-13T23:42:41.309 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e74197700 1 -- start start 2024-09-13T23:42:41.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e74197700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f9e6c095d50 0x7f9e6c12efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:41.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e74197700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6c06cbf0 con 0x7f9e6c095260 2024-09-13T23:42:41.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e74197700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6c04f000 con 0x7f9e6c095d50 2024-09-13T23:42:41.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e71f33700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f9e6c095260 0x7f9e6c095640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33754/0 (socket says 172.21.15.12:33754) 2024-09-13T23:42:41.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e71f33700 1 -- 172.21.15.12:0/873306820 learned_addr learned my addr 172.21.15.12:0/873306820 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:42:41.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e71732700 1 --2- 172.21.15.12:0/873306820 >> v2:172.21.15.12:3300/0 conn(0x7f9e6c095d50 0x7f9e6c12efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:41.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e70f31700 1 -- 172.21.15.12:0/873306820 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3681639861 0 0) 0x7f9e6c06cbf0 con 0x7f9e6c095260 2024-09-13T23:42:41.311 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e70f31700 1 -- 172.21.15.12:0/873306820 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e580035d0 con 0x7f9e6c095260 2024-09-13T23:42:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e70f31700 1 -- 172.21.15.12:0/873306820 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613442682 0 0) 0x7f9e580035d0 con 0x7f9e6c095260 2024-09-13T23:42:41.312 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e70f31700 1 -- 172.21.15.12:0/873306820 >> v2:172.21.15.12:3300/0 conn(0x7f9e6c095d50 msgr2=0x7f9e6c12efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T23:42:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e70f31700 1 --2- 172.21.15.12:0/873306820 >> v2:172.21.15.12:3300/0 conn(0x7f9e6c095d50 0x7f9e6c12efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e70f31700 1 -- 172.21.15.12:0/873306820 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c06ae20 con 0x7f9e6c095260 2024-09-13T23:42:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.305+0000 7f9e70f31700 1 -- 172.21.15.12:0/873306820 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 222865856 0 0) 0x7f9e680015a0 con 0x7f9e6c095260 2024-09-13T23:42:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e70f31700 1 -- 172.21.15.12:0/873306820 >> v1:172.21.15.12:6789/0 conn(0x7f9e6c095260 legacy=0x7f9e6c095640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e70f31700 1 --2- 172.21.15.12:0/873306820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 0x7f9e58003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e70f31700 1 -- 172.21.15.12:0/873306820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580044e0 con 0x7f9e58003a20 2024-09-13T23:42:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e71f33700 1 --2- 172.21.15.12:0/873306820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 0x7f9e58003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e71f33700 1 -- 172.21.15.12:0/873306820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580035d0 con 0x7f9e58003a20 2024-09-13T23:42:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e71f33700 1 --2- 172.21.15.12:0/873306820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 0x7f9e58003e20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800c1c0 tx=0x7f9e580058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0313a541f169a33 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:41.313 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e70f31700 1 -- 172.21.15.12:0/873306820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e6800cbb0 con 0x7f9e58003a20 2024-09-13T23:42:41.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e70f31700 1 -- 172.21.15.12:0/873306820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6800cd10 con 0x7f9e58003a20 2024-09-13T23:42:41.314 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e70f31700 1 -- 172.21.15.12:0/873306820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e68005980 con 0x7f9e58003a20 2024-09-13T23:42:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e74197700 1 -- 172.21.15.12:0/873306820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 msgr2=0x7f9e58003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e74197700 1 --2- 172.21.15.12:0/873306820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 0x7f9e58003e20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800c1c0 tx=0x7f9e580058a0 comp rx=0 tx=0).stop 2024-09-13T23:42:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e74197700 1 -- 172.21.15.12:0/873306820 shutdown_connections 2024-09-13T23:42:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e74197700 1 --2- 172.21.15.12:0/873306820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 0x7f9e58003e20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e74197700 1 --2- 172.21.15.12:0/873306820 >> v2:172.21.15.12:3300/0 conn(0x7f9e6c095d50 0x7f9e6c12efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e74197700 1 -- 172.21.15.12:0/873306820 >> 172.21.15.12:0/873306820 conn(0x7f9e6c0986d0 msgr2=0x7f9e6c098ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e74197700 1 -- 172.21.15.12:0/873306820 shutdown_connections 2024-09-13T23:42:41.315 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.309+0000 7f9e74197700 1 -- 172.21.15.12:0/873306820 wait complete. 2024-09-13T23:42:41.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e74197700 1 Processor -- start 2024-09-13T23:42:41.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e74197700 1 -- start start 2024-09-13T23:42:41.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e74197700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 0x7f9e6c12e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:41.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e74197700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c091e20 con 0x7f9e58003a20 2024-09-13T23:42:41.316 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e71f33700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 0x7f9e6c12e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:41.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e71f33700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 0x7f9e6c12e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44836/0 (socket says 172.21.15.12:44836) 2024-09-13T23:42:41.317 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e71f33700 1 -- 172.21.15.12:0/3386440331 learned_addr learned my addr 172.21.15.12:0/3386440331 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:42:41.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e71f33700 1 -- 172.21.15.12:0/3386440331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c0718c0 con 0x7f9e58003a20 2024-09-13T23:42:41.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e71f33700 1 --2- 172.21.15.12:0/3386440331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 0x7f9e6c12e130 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9e68001eb0 tx=0x7f9e6c0717f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:41.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e627fc700 1 -- 172.21.15.12:0/3386440331 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e68001560 con 0x7f9e58003a20 2024-09-13T23:42:41.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e627fc700 1 -- 172.21.15.12:0/3386440331 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e680070d0 con 0x7f9e58003a20 2024-09-13T23:42:41.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e74197700 1 -- 172.21.15.12:0/3386440331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6c090ca0 con 0x7f9e58003a20 2024-09-13T23:42:41.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e627fc700 1 -- 172.21.15.12:0/3386440331 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e68007370 con 0x7f9e58003a20 2024-09-13T23:42:41.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e74197700 1 -- 172.21.15.12:0/3386440331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c094330 con 0x7f9e58003a20 2024-09-13T23:42:41.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e627fc700 1 -- 172.21.15.12:0/3386440331 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f9e68026450 con 0x7f9e58003a20 2024-09-13T23:42:41.318 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.313+0000 7f9e627fc700 1 -- 172.21.15.12:0/3386440331 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f9e68026c10 con 0x7f9e58003a20 2024-09-13T23:42:41.319 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.317+0000 7f9e74197700 1 -- 172.21.15.12:0/3386440331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6c091c30 con 0x7f9e58003a20 2024-09-13T23:42:41.321 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.317+0000 7f9e627fc700 1 -- 172.21.15.12:0/3386440331 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f9e6802b060 con 0x7f9e58003a20 2024-09-13T23:42:41.390 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.385+0000 7f9e74197700 1 -- 172.21.15.12:0/3386440331 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e6c091f60 con 0x7f9e58003a20 2024-09-13T23:42:41.391 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.389+0000 7f9e627fc700 1 -- 172.21.15.12:0/3386440331 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4152 (secure 0 0 0) 0x7f9e68021070 con 0x7f9e58003a20 2024-09-13T23:42:41.391 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:41.391 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":6,"fsid":"dc64727a-88fa-459b-b2f7-e6853bb706a5","created":"2024-09-13T23:42:40.040576+0000","modified":"2024-09-13T23:42:41.150993+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T23:42:41.150993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e2f894d3-5bb4-46dd-9be1-bbae3cddb090","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"840a539a-6bbb-4ccd-a32c-7bdf4e7e339c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b370fb03-da40-4119-8bc0-6caff41336a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cf2a8b98-93cb-4290-a41d-c833ba4c647c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T23:42:41.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.389+0000 7f9e74197700 1 -- 172.21.15.12:0/3386440331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 msgr2=0x7f9e6c12e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:41.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.389+0000 7f9e74197700 1 --2- 172.21.15.12:0/3386440331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 0x7f9e6c12e130 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9e68001eb0 tx=0x7f9e6c0717f0 comp rx=0 tx=0).stop 2024-09-13T23:42:41.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.389+0000 7f9e74197700 1 -- 172.21.15.12:0/3386440331 shutdown_connections 2024-09-13T23:42:41.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.389+0000 7f9e74197700 1 --2- 172.21.15.12:0/3386440331 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f9e58003a20 0x7f9e6c12e130 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:41.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.389+0000 7f9e74197700 1 -- 172.21.15.12:0/3386440331 >> 172.21.15.12:0/3386440331 conn(0x7f9e6c0986d0 msgr2=0x7f9e6c097460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:41.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.389+0000 7f9e74197700 1 -- 172.21.15.12:0/3386440331 shutdown_connections 2024-09-13T23:42:41.393 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:41.389+0000 7f9e74197700 1 -- 172.21.15.12:0/3386440331 wait complete. 2024-09-13T23:42:41.404 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T23:42:41.404 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T23:42:41.519 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:41.512+0000 7f3b562152c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T23:42:41.551 INFO:tasks.ceph.osd.1.smithi167.stderr:2024-09-13T23:42:41.544+0000 7fedaaed2880 -1 Falling back to public interface 2024-09-13T23:42:41.572 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:41.564+0000 7f3b562152c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T23:42:41.770 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:41.764+0000 7f3b562152c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T23:42:41.791 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-09-13T23:42:41.784+0000 7fe75ad7f880 -1 Falling back to public interface 2024-09-13T23:42:41.808 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T23:42:41.809 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-13T23:42:41.805+0000 7f3ea8e8f880 -1 Falling back to public interface 2024-09-13T23:42:41.823 INFO:tasks.ceph.osd.3.smithi167.stderr:2024-09-13T23:42:41.816+0000 7f46913fd880 -1 Falling back to public interface 2024-09-13T23:42:41.839 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:41.832+0000 7f3b562152c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T23:42:41.908 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:41.900+0000 7f3b562152c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T23:42:41.918 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3bede80700 1 Processor -- start 2024-09-13T23:42:41.918 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3bede80700 1 -- start start 2024-09-13T23:42:41.918 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3bede80700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3be809c9a0 0x7f3be8132220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:41.918 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3bede80700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be8099710 con 0x7f3be812ccf0 2024-09-13T23:42:41.918 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3bede80700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be8098bf0 con 0x7f3be809c9a0 2024-09-13T23:42:41.918 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3be7fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3be809c9a0 0x7f3be8132220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:41.918 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3be7fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3be809c9a0 0x7f3be8132220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44840/0 (socket says 172.21.15.12:44840) 2024-09-13T23:42:41.918 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3be7fff700 1 -- 172.21.15.12:0/2065296123 learned_addr learned my addr 172.21.15.12:0/2065296123 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:42:41.918 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3bece7e700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f3be812ccf0 0x7f3be812d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:33766/0 (socket says 172.21.15.12:33766) 2024-09-13T23:42:41.920 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3be7fff700 1 -- 172.21.15.12:0/2065296123 >> v1:172.21.15.12:6789/0 conn(0x7f3be812ccf0 legacy=0x7f3be812d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:41.920 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3be7fff700 1 -- 172.21.15.12:0/2065296123 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be812d7e0 con 0x7f3be809c9a0 2024-09-13T23:42:41.920 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3be7fff700 1 --2- 172.21.15.12:0/2065296123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3be809c9a0 0x7f3be8132220 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009e50 tx=0x7f3bd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a44f155ff9a540 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:41.920 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.913+0000 7f3be77fe700 1 -- 172.21.15.12:0/2065296123 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bd400e070 con 0x7f3be809c9a0 2024-09-13T23:42:41.920 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3be77fe700 1 -- 172.21.15.12:0/2065296123 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd4003a40 con 0x7f3be809c9a0 2024-09-13T23:42:41.920 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3be77fe700 1 -- 172.21.15.12:0/2065296123 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bd4003d00 con 0x7f3be809c9a0 2024-09-13T23:42:41.920 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bede80700 1 -- 172.21.15.12:0/2065296123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3be809c9a0 msgr2=0x7f3be8132220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:41.920 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bede80700 1 --2- 172.21.15.12:0/2065296123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3be809c9a0 0x7f3be8132220 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009e50 tx=0x7f3bd400b330 comp rx=0 tx=0).stop 2024-09-13T23:42:41.920 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bede80700 1 -- 172.21.15.12:0/2065296123 shutdown_connections 2024-09-13T23:42:41.920 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bede80700 1 --2- 172.21.15.12:0/2065296123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3be809c9a0 0x7f3be8132220 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:41.921 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bede80700 1 -- 172.21.15.12:0/2065296123 >> 172.21.15.12:0/2065296123 conn(0x7f3be809fbe0 msgr2=0x7f3be809c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:41.921 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bede80700 1 -- 172.21.15.12:0/2065296123 shutdown_connections 2024-09-13T23:42:41.921 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bede80700 1 -- 172.21.15.12:0/2065296123 wait complete. 2024-09-13T23:42:41.921 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bede80700 1 Processor -- start 2024-09-13T23:42:41.921 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bede80700 1 -- start start 2024-09-13T23:42:41.922 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bede80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3be809c9a0 0x7f3be81c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:41.922 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bede80700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be806d8f0 con 0x7f3be809c9a0 2024-09-13T23:42:41.922 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bece7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3be809c9a0 0x7f3be81c1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:41.922 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bece7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3be809c9a0 0x7f3be81c1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:44848/0 (socket says 172.21.15.12:44848) 2024-09-13T23:42:41.922 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bece7e700 1 -- 172.21.15.12:0/3070971030 learned_addr learned my addr 172.21.15.12:0/3070971030 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:42:41.923 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.917+0000 7f3bece7e700 1 -- 172.21.15.12:0/3070971030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd4009b30 con 0x7f3be809c9a0 2024-09-13T23:42:41.923 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.921+0000 7f3bece7e700 1 --2- 172.21.15.12:0/3070971030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3be809c9a0 0x7f3be81c1c90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800c480 tx=0x7f3bd800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:41.924 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.921+0000 7f3be57fa700 1 -- 172.21.15.12:0/3070971030 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bd8005c00 con 0x7f3be809c9a0 2024-09-13T23:42:41.924 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.921+0000 7f3be57fa700 1 -- 172.21.15.12:0/3070971030 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd8005d60 con 0x7f3be809c9a0 2024-09-13T23:42:41.924 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.921+0000 7f3bede80700 1 -- 172.21.15.12:0/3070971030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be806d5c0 con 0x7f3be809c9a0 2024-09-13T23:42:41.924 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.921+0000 7f3be57fa700 1 -- 172.21.15.12:0/3070971030 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3bd80070d0 con 0x7f3be809c9a0 2024-09-13T23:42:41.924 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.921+0000 7f3bede80700 1 -- 172.21.15.12:0/3070971030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be81276c0 con 0x7f3be809c9a0 2024-09-13T23:42:41.925 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.921+0000 7f3be57fa700 1 -- 172.21.15.12:0/3070971030 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f3bd8007600 con 0x7f3be809c9a0 2024-09-13T23:42:41.925 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.921+0000 7f3bede80700 1 -- 172.21.15.12:0/3070971030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be80987d0 con 0x7f3be809c9a0 2024-09-13T23:42:41.925 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.921+0000 7f3be57fa700 1 -- 172.21.15.12:0/3070971030 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f3bd801b8c0 con 0x7f3be809c9a0 2024-09-13T23:42:41.928 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.925+0000 7f3be57fa700 1 -- 172.21.15.12:0/3070971030 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f3bd8021060 con 0x7f3be809c9a0 2024-09-13T23:42:41.996 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.993+0000 7f3bede80700 1 -- 172.21.15.12:0/3070971030 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3be812db50 con 0x7f3be809c9a0 2024-09-13T23:42:41.997 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.993+0000 7f3be57fa700 1 -- 172.21.15.12:0/3070971030 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4152 (secure 0 0 0) 0x7f3bd8017070 con 0x7f3be809c9a0 2024-09-13T23:42:41.997 INFO:teuthology.misc.health.smithi012.stdout: 2024-09-13T23:42:41.997 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":6,"fsid":"dc64727a-88fa-459b-b2f7-e6853bb706a5","created":"2024-09-13T23:42:40.040576+0000","modified":"2024-09-13T23:42:41.150993+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T23:42:41.150993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e2f894d3-5bb4-46dd-9be1-bbae3cddb090","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"840a539a-6bbb-4ccd-a32c-7bdf4e7e339c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b370fb03-da40-4119-8bc0-6caff41336a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cf2a8b98-93cb-4290-a41d-c833ba4c647c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T23:42:41.998 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.993+0000 7f3bede80700 1 -- 172.21.15.12:0/3070971030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3be809c9a0 msgr2=0x7f3be81c1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:41.998 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.993+0000 7f3bede80700 1 --2- 172.21.15.12:0/3070971030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3be809c9a0 0x7f3be81c1c90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800c480 tx=0x7f3bd800c840 comp rx=0 tx=0).stop 2024-09-13T23:42:41.998 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.993+0000 7f3bede80700 1 -- 172.21.15.12:0/3070971030 shutdown_connections 2024-09-13T23:42:41.998 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.993+0000 7f3bede80700 1 --2- 172.21.15.12:0/3070971030 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3be809c9a0 0x7f3be81c1c90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:41.998 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.993+0000 7f3bede80700 1 -- 172.21.15.12:0/3070971030 >> 172.21.15.12:0/3070971030 conn(0x7f3be809fbe0 msgr2=0x7f3be8126680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:41.999 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.993+0000 7f3bede80700 1 -- 172.21.15.12:0/3070971030 shutdown_connections 2024-09-13T23:42:41.999 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:41.993+0000 7f3bede80700 1 -- 172.21.15.12:0/3070971030 wait complete. 2024-09-13T23:42:42.010 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-09-13T23:42:42.334 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:42.328+0000 7f3b562152c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T23:42:42.461 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:42.452+0000 7f3b562152c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T23:42:42.825 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:42.816+0000 7f3b562152c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T23:42:43.013 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:43.004+0000 7f3b562152c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T23:42:43.075 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:43.068+0000 7f3b562152c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T23:42:43.236 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:43.228+0000 7f3b562152c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T23:42:43.959 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:43.952+0000 7f3b562152c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T23:42:44.026 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:44.020+0000 7f3b562152c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T23:42:44.177 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:44.172+0000 7f3b562152c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T23:42:44.245 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:44.236+0000 7f3b562152c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T23:42:44.317 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:44.308+0000 7f3b562152c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T23:42:44.657 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:44.648+0000 7f3b562152c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T23:42:44.735 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:44.728+0000 7f3b562152c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T23:42:45.115 INFO:tasks.ceph.osd.1.smithi167.stderr:2024-09-13T23:42:45.108+0000 7fedaaed2880 -1 osd.1 0 log_to_monitors true 2024-09-13T23:42:45.391 INFO:tasks.ceph.osd.3.smithi167.stderr:2024-09-13T23:42:45.384+0000 7f46913fd880 -1 osd.3 0 log_to_monitors true 2024-09-13T23:42:45.429 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:45.420+0000 7f3b562152c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T23:42:45.530 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-13T23:42:45.524+0000 7f3ea8e8f880 -1 osd.0 0 log_to_monitors true 2024-09-13T23:42:45.567 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-09-13T23:42:45.560+0000 7fe75ad7f880 -1 osd.2 0 log_to_monitors true 2024-09-13T23:42:46.466 INFO:tasks.ceph.mgr.x.smithi167.stderr:2024-09-13T23:42:46.460+0000 7f3b3cd89700 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-13T23:42:47.480 INFO:tasks.ceph.osd.1.smithi167.stderr:2024-09-13T23:42:47.472+0000 7feda7e99700 -1 osd.1 0 waiting for initial osdmap 2024-09-13T23:42:47.502 INFO:tasks.ceph.osd.1.smithi167.stderr:2024-09-13T23:42:47.496+0000 7fed9e7c3700 -1 osd.1 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T23:42:48.414 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T23:42:48.428 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-13T23:42:48.424+0000 7f3ea44ab700 -1 osd.0 0 waiting for initial osdmap 2024-09-13T23:42:48.432 INFO:tasks.ceph.osd.0.smithi012.stderr:2024-09-13T23:42:48.428+0000 7f3e9c780700 -1 osd.0 9 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T23:42:48.513 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.508+0000 7efffd171700 1 Processor -- start 2024-09-13T23:42:48.514 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.508+0000 7efffd171700 1 -- start start 2024-09-13T23:42:48.515 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.508+0000 7efffd171700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efff806ebc0 0x7efff806efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:48.516 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.508+0000 7efffd171700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff8070470 con 0x7efff8066650 2024-09-13T23:42:48.516 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.508+0000 7efffd171700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efff806b3b0 con 0x7efff806ebc0 2024-09-13T23:42:48.516 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.508+0000 7efff7fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efff806ebc0 0x7efff806efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:48.516 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.508+0000 7efff7fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efff806ebc0 0x7efff806efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39590/0 (socket says 172.21.15.12:39590) 2024-09-13T23:42:48.516 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.508+0000 7efff7fff700 1 -- 172.21.15.12:0/2188628458 learned_addr learned my addr 172.21.15.12:0/2188628458 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:42:48.516 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.508+0000 7efff77fe700 1 --1- 172.21.15.12:0/2188628458 >> v1:172.21.15.12:6789/0 conn(0x7efff8066650 0x7efff8095060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59634/0 (socket says 172.21.15.12:59634) 2024-09-13T23:42:48.516 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efff7fff700 1 -- 172.21.15.12:0/2188628458 >> v1:172.21.15.12:6789/0 conn(0x7efff8066650 legacy=0x7efff8095060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:48.516 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efff7fff700 1 -- 172.21.15.12:0/2188628458 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff8070890 con 0x7efff806ebc0 2024-09-13T23:42:48.516 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efff7fff700 1 --2- 172.21.15.12:0/2188628458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff806ebc0 0x7efff806efa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7effe80098c0 tx=0x7effe8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd848ccf3c620422 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:48.516 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efff6ffd700 1 -- 172.21.15.12:0/2188628458 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effe8004160 con 0x7efff806ebc0 2024-09-13T23:42:48.516 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efff6ffd700 1 -- 172.21.15.12:0/2188628458 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe80042c0 con 0x7efff806ebc0 2024-09-13T23:42:48.516 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efff6ffd700 1 -- 172.21.15.12:0/2188628458 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effe8004580 con 0x7efff806ebc0 2024-09-13T23:42:48.517 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efffd171700 1 -- 172.21.15.12:0/2188628458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff806ebc0 msgr2=0x7efff806efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:48.517 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efffd171700 1 --2- 172.21.15.12:0/2188628458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff806ebc0 0x7efff806efa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7effe80098c0 tx=0x7effe8009bd0 comp rx=0 tx=0).stop 2024-09-13T23:42:48.518 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efffd171700 1 -- 172.21.15.12:0/2188628458 shutdown_connections 2024-09-13T23:42:48.518 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efffd171700 1 --2- 172.21.15.12:0/2188628458 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff806ebc0 0x7efff806efa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:48.518 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efffd171700 1 -- 172.21.15.12:0/2188628458 >> 172.21.15.12:0/2188628458 conn(0x7efff8098c00 msgr2=0x7efff806c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:48.518 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efffd171700 1 -- 172.21.15.12:0/2188628458 shutdown_connections 2024-09-13T23:42:48.518 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.512+0000 7efffd171700 1 -- 172.21.15.12:0/2188628458 wait complete. 2024-09-13T23:42:48.519 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efffd171700 1 Processor -- start 2024-09-13T23:42:48.519 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efffd171700 1 -- start start 2024-09-13T23:42:48.520 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efffd171700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8066650 0x7efff81290b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:48.521 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efffd171700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff8094f60 con 0x7efff8066650 2024-09-13T23:42:48.521 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efff7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8066650 0x7efff81290b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:48.521 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efff7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8066650 0x7efff81290b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39606/0 (socket says 172.21.15.12:39606) 2024-09-13T23:42:48.521 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efff7fff700 1 -- 172.21.15.12:0/1371548024 learned_addr learned my addr 172.21.15.12:0/1371548024 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:42:48.522 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efff7fff700 1 -- 172.21.15.12:0/1371548024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe8009570 con 0x7efff8066650 2024-09-13T23:42:48.522 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efff7fff700 1 --2- 172.21.15.12:0/1371548024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8066650 0x7efff81290b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efff8067c00 tx=0x7effe8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:48.522 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efff4ff9700 1 -- 172.21.15.12:0/1371548024 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effe8011630 con 0x7efff8066650 2024-09-13T23:42:48.522 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efff4ff9700 1 -- 172.21.15.12:0/1371548024 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe80117b0 con 0x7efff8066650 2024-09-13T23:42:48.522 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efffd171700 1 -- 172.21.15.12:0/1371548024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff8089670 con 0x7efff8066650 2024-09-13T23:42:48.522 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efff4ff9700 1 -- 172.21.15.12:0/1371548024 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effe8011a90 con 0x7efff8066650 2024-09-13T23:42:48.522 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.516+0000 7efffd171700 1 -- 172.21.15.12:0/1371548024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff806dd20 con 0x7efff8066650 2024-09-13T23:42:48.523 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.520+0000 7efff4ff9700 1 -- 172.21.15.12:0/1371548024 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 4) v1 ==== 43251+0+0 (secure 0 0 0) 0x7effe8021070 con 0x7efff8066650 2024-09-13T23:42:48.523 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.520+0000 7efffd171700 1 -- 172.21.15.12:0/1371548024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff8089120 con 0x7efff8066650 2024-09-13T23:42:48.523 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.520+0000 7efff4ff9700 1 --2- 172.21.15.12:0/1371548024 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7effe40354b0 0x7effe4037970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:48.524 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.520+0000 7efff4ff9700 1 -- 172.21.15.12:0/1371548024 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2183+0+0 (secure 0 0 0) 0x7effe8048be0 con 0x7efff8066650 2024-09-13T23:42:48.524 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.520+0000 7efff77fe700 1 --2- 172.21.15.12:0/1371548024 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7effe40354b0 0x7effe4037970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:48.524 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.520+0000 7efff77fe700 1 --2- 172.21.15.12:0/1371548024 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7effe40354b0 0x7effe4037970 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7effec000f60 tx=0x7effec007b30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:48.533 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.528+0000 7efff4ff9700 1 -- 172.21.15.12:0/1371548024 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7effe8019d90 con 0x7efff8066650 2024-09-13T23:42:48.693 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-09-13T23:42:48.688+0000 7fe75639b700 -1 osd.2 0 waiting for initial osdmap 2024-09-13T23:42:48.698 INFO:tasks.ceph.osd.2.smithi167.stderr:2024-09-13T23:42:48.692+0000 7fe74e670700 -1 osd.2 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T23:42:48.705 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.700+0000 7efffd171700 1 -- 172.21.15.12:0/1371548024 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efff80008d0 con 0x7efff8066650 2024-09-13T23:42:48.705 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.700+0000 7efff4ff9700 1 -- 172.21.15.12:0/1371548024 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+5136 (secure 0 0 0) 0x7effe8019bb0 con 0x7efff8066650 2024-09-13T23:42:48.705 INFO:teuthology.misc.health.smithi012.stdout: 2024-09-13T23:42:48.706 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":10,"fsid":"dc64727a-88fa-459b-b2f7-e6853bb706a5","created":"2024-09-13T23:42:40.040576+0000","modified":"2024-09-13T23:42:48.481048+0000","last_up_change":"2024-09-13T23:42:48.481048+0000","last_in_change":"2024-09-13T23:42:41.150993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e2f894d3-5bb4-46dd-9be1-bbae3cddb090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6801","nonce":2649399468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6803","nonce":2649399468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6807","nonce":2649399468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6805","nonce":2649399468}]},"public_addr":"172.21.15.12:6801/2649399468","cluster_addr":"172.21.15.12:6803/2649399468","heartbeat_back_addr":"172.21.15.12:6807/2649399468","heartbeat_front_addr":"172.21.15.12:6805/2649399468","state":["exists","up"]},{"osd":1,"uuid":"840a539a-6bbb-4ccd-a32c-7bdf4e7e339c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6801","nonce":3875661733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6803","nonce":3875661733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6807","nonce":3875661733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6805","nonce":3875661733}]},"public_addr":"172.21.15.167:6801/3875661733","cluster_addr":"172.21.15.167:6803/3875661733","heartbeat_back_addr":"172.21.15.167:6807/3875661733","heartbeat_front_addr":"172.21.15.167:6805/3875661733","state":["exists","up"]},{"osd":2,"uuid":"b370fb03-da40-4119-8bc0-6caff41336a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"cf2a8b98-93cb-4290-a41d-c833ba4c647c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T23:42:48.708 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.704+0000 7efffd171700 1 -- 172.21.15.12:0/1371548024 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7effe40354b0 msgr2=0x7effe4037970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:48.708 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.704+0000 7efffd171700 1 --2- 172.21.15.12:0/1371548024 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7effe40354b0 0x7effe4037970 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7effec000f60 tx=0x7effec007b30 comp rx=0 tx=0).stop 2024-09-13T23:42:48.708 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.704+0000 7efffd171700 1 -- 172.21.15.12:0/1371548024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8066650 msgr2=0x7efff81290b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:48.708 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.704+0000 7efffd171700 1 --2- 172.21.15.12:0/1371548024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8066650 0x7efff81290b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efff8067c00 tx=0x7effe8004770 comp rx=0 tx=0).stop 2024-09-13T23:42:48.709 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.704+0000 7efffd171700 1 -- 172.21.15.12:0/1371548024 shutdown_connections 2024-09-13T23:42:48.709 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.704+0000 7efffd171700 1 --2- 172.21.15.12:0/1371548024 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7effe40354b0 0x7effe4037970 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:48.709 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.704+0000 7efffd171700 1 --2- 172.21.15.12:0/1371548024 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efff8066650 0x7efff81290b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:48.709 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.704+0000 7efffd171700 1 -- 172.21.15.12:0/1371548024 >> 172.21.15.12:0/1371548024 conn(0x7efff8098c00 msgr2=0x7efff81b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:48.709 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.704+0000 7efffd171700 1 -- 172.21.15.12:0/1371548024 shutdown_connections 2024-09-13T23:42:48.709 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:48.704+0000 7efffd171700 1 -- 172.21.15.12:0/1371548024 wait complete. 2024-09-13T23:42:48.719 DEBUG:teuthology.misc:2 of 4 OSDs are up 2024-09-13T23:42:48.751 INFO:tasks.ceph.osd.3.smithi167.stderr:2024-09-13T23:42:48.744+0000 7f468ca19700 -1 osd.3 0 waiting for initial osdmap 2024-09-13T23:42:48.755 INFO:tasks.ceph.osd.3.smithi167.stderr:2024-09-13T23:42:48.748+0000 7f4684cee700 -1 osd.3 10 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T23:42:55.123 DEBUG:teuthology.orchestra.run.smithi012:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T23:42:55.234 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.228+0000 7f4f2f3c0700 1 Processor -- start 2024-09-13T23:42:55.234 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.228+0000 7f4f2f3c0700 1 -- start start 2024-09-13T23:42:55.235 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.228+0000 7f4f2f3c0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4f28095900 0x7f4f2806da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:55.235 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.228+0000 7f4f2f3c0700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f2806f1f0 con 0x7f4f2806a600 2024-09-13T23:42:55.235 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.228+0000 7f4f2f3c0700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f280717d0 con 0x7f4f28095900 2024-09-13T23:42:55.235 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.228+0000 7f4f2e3be700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4f28095900 0x7f4f2806da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:55.235 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.228+0000 7f4f2dbbd700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4f2806a600 0x7f4f2806df90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59650/0 (socket says 172.21.15.12:59650) 2024-09-13T23:42:55.235 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.228+0000 7f4f2dbbd700 1 -- 172.21.15.12:0/3117200767 learned_addr learned my addr 172.21.15.12:0/3117200767 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:42:55.236 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2e3be700 1 -- 172.21.15.12:0/3117200767 >> v1:172.21.15.12:6789/0 conn(0x7f4f2806a600 legacy=0x7f4f2806df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:55.236 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2e3be700 1 -- 172.21.15.12:0/3117200767 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f2806edf0 con 0x7f4f28095900 2024-09-13T23:42:55.236 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2e3be700 1 --2- 172.21.15.12:0/3117200767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f28095900 0x7f4f2806da50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009800 tx=0x7f4f24009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4116b16b6d923ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:55.236 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2d3bc700 1 -- 172.21.15.12:0/3117200767 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f24004160 con 0x7f4f28095900 2024-09-13T23:42:55.236 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2d3bc700 1 -- 172.21.15.12:0/3117200767 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f240042c0 con 0x7f4f28095900 2024-09-13T23:42:55.236 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2d3bc700 1 -- 172.21.15.12:0/3117200767 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f24004580 con 0x7f4f28095900 2024-09-13T23:42:55.238 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3117200767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f28095900 msgr2=0x7f4f2806da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:55.238 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2f3c0700 1 --2- 172.21.15.12:0/3117200767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f28095900 0x7f4f2806da50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f24009800 tx=0x7f4f24009b10 comp rx=0 tx=0).stop 2024-09-13T23:42:55.238 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3117200767 shutdown_connections 2024-09-13T23:42:55.238 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2f3c0700 1 --2- 172.21.15.12:0/3117200767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f28095900 0x7f4f2806da50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:55.238 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3117200767 >> 172.21.15.12:0/3117200767 conn(0x7f4f28098c00 msgr2=0x7f4f2806c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:55.238 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3117200767 shutdown_connections 2024-09-13T23:42:55.238 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.232+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3117200767 wait complete. 2024-09-13T23:42:55.239 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f2f3c0700 1 Processor -- start 2024-09-13T23:42:55.239 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f2f3c0700 1 -- start start 2024-09-13T23:42:55.240 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f2f3c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2806a600 0x7f4f281be2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:55.240 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f2f3c0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f281a7610 con 0x7f4f2806a600 2024-09-13T23:42:55.240 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f2e3be700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2806a600 0x7f4f281be2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:55.240 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f2e3be700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2806a600 0x7f4f281be2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39628/0 (socket says 172.21.15.12:39628) 2024-09-13T23:42:55.240 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f2e3be700 1 -- 172.21.15.12:0/3974556568 learned_addr learned my addr 172.21.15.12:0/3974556568 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:42:55.240 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f2e3be700 1 -- 172.21.15.12:0/3974556568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f240094b0 con 0x7f4f2806a600 2024-09-13T23:42:55.241 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f2e3be700 1 --2- 172.21.15.12:0/3974556568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2806a600 0x7f4f281be2e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f240090b0 tx=0x7f4f24004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:55.241 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f1effd700 1 -- 172.21.15.12:0/3974556568 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f24006ea0 con 0x7f4f2806a600 2024-09-13T23:42:55.241 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f1effd700 1 -- 172.21.15.12:0/3974556568 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2401f070 con 0x7f4f2806a600 2024-09-13T23:42:55.241 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f1effd700 1 -- 172.21.15.12:0/3974556568 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4f24011630 con 0x7f4f2806a600 2024-09-13T23:42:55.242 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3974556568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f281a9710 con 0x7f4f2806a600 2024-09-13T23:42:55.242 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3974556568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f281a9a20 con 0x7f4f2806a600 2024-09-13T23:42:55.242 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f1effd700 1 -- 172.21.15.12:0/3974556568 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f4f24021030 con 0x7f4f2806a600 2024-09-13T23:42:55.242 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f1effd700 1 --2- 172.21.15.12:0/3974556568 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f4f0c0354f0 0x7f4f0c0379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:55.242 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3974556568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f28067600 con 0x7f4f2806a600 2024-09-13T23:42:55.243 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.236+0000 7f4f1effd700 1 -- 172.21.15.12:0/3974556568 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3129+0+0 (secure 0 0 0) 0x7f4f24048d40 con 0x7f4f2806a600 2024-09-13T23:42:55.243 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.240+0000 7f4f2dbbd700 1 --2- 172.21.15.12:0/3974556568 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f4f0c0354f0 0x7f4f0c0379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:55.244 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.240+0000 7f4f2dbbd700 1 --2- 172.21.15.12:0/3974556568 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f4f0c0354f0 0x7f4f0c0379b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4f180016a0 tx=0x7f4f18007b70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:55.249 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.244+0000 7f4f1effd700 1 -- 172.21.15.12:0/3974556568 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4f24019d90 con 0x7f4f2806a600 2024-09-13T23:42:55.420 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.416+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3974556568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f2806da50 con 0x7f4f2806a600 2024-09-13T23:42:55.421 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.416+0000 7f4f1effd700 1 -- 172.21.15.12:0/3974556568 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+7957 (secure 0 0 0) 0x7f4f2401b440 con 0x7f4f2806a600 2024-09-13T23:42:55.421 INFO:teuthology.misc.health.smithi012.stdout: 2024-09-13T23:42:55.422 INFO:teuthology.misc.health.smithi012.stdout:{"epoch":14,"fsid":"dc64727a-88fa-459b-b2f7-e6853bb706a5","created":"2024-09-13T23:42:40.040576+0000","modified":"2024-09-13T23:42:52.496663+0000","last_up_change":"2024-09-13T23:42:49.482698+0000","last_in_change":"2024-09-13T23:42:41.150993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T23:42:49.531017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2f894d3-5bb4-46dd-9be1-bbae3cddb090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6801","nonce":2649399468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6803","nonce":2649399468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6807","nonce":2649399468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6805","nonce":2649399468}]},"public_addr":"172.21.15.12:6801/2649399468","cluster_addr":"172.21.15.12:6803/2649399468","heartbeat_back_addr":"172.21.15.12:6807/2649399468","heartbeat_front_addr":"172.21.15.12:6805/2649399468","state":["exists","up"]},{"osd":1,"uuid":"840a539a-6bbb-4ccd-a32c-7bdf4e7e339c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6801","nonce":3875661733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6803","nonce":3875661733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6807","nonce":3875661733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6805","nonce":3875661733}]},"public_addr":"172.21.15.167:6801/3875661733","cluster_addr":"172.21.15.167:6803/3875661733","heartbeat_back_addr":"172.21.15.167:6807/3875661733","heartbeat_front_addr":"172.21.15.167:6805/3875661733","state":["exists","up"]},{"osd":2,"uuid":"b370fb03-da40-4119-8bc0-6caff41336a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1180369139},{"type":"v1","addr":"172.21.15.167:6809","nonce":1180369139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1180369139},{"type":"v1","addr":"172.21.15.167:6811","nonce":1180369139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1180369139},{"type":"v1","addr":"172.21.15.167:6815","nonce":1180369139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1180369139},{"type":"v1","addr":"172.21.15.167:6813","nonce":1180369139}]},"public_addr":"172.21.15.167:6809/1180369139","cluster_addr":"172.21.15.167:6811/1180369139","heartbeat_back_addr":"172.21.15.167:6815/1180369139","heartbeat_front_addr":"172.21.15.167:6813/1180369139","state":["exists","up"]},{"osd":3,"uuid":"cf2a8b98-93cb-4290-a41d-c833ba4c647c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3688363524},{"type":"v1","addr":"172.21.15.167:6817","nonce":3688363524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3688363524},{"type":"v1","addr":"172.21.15.167:6819","nonce":3688363524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3688363524},{"type":"v1","addr":"172.21.15.167:6823","nonce":3688363524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3688363524},{"type":"v1","addr":"172.21.15.167:6821","nonce":3688363524}]},"public_addr":"172.21.15.167:6817/3688363524","cluster_addr":"172.21.15.167:6819/3688363524","heartbeat_back_addr":"172.21.15.167:6823/3688363524","heartbeat_front_addr":"172.21.15.167:6821/3688363524","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T23:42:46.484652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T23:42:46.146279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T23:42:46.609332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T23:42:46.363369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T23:42:55.424 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.420+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3974556568 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f4f0c0354f0 msgr2=0x7f4f0c0379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:55.424 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.420+0000 7f4f2f3c0700 1 --2- 172.21.15.12:0/3974556568 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f4f0c0354f0 0x7f4f0c0379b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4f180016a0 tx=0x7f4f18007b70 comp rx=0 tx=0).stop 2024-09-13T23:42:55.424 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.420+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3974556568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2806a600 msgr2=0x7f4f281be2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:55.424 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.420+0000 7f4f2f3c0700 1 --2- 172.21.15.12:0/3974556568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2806a600 0x7f4f281be2e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f240090b0 tx=0x7f4f24004770 comp rx=0 tx=0).stop 2024-09-13T23:42:55.424 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.420+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3974556568 shutdown_connections 2024-09-13T23:42:55.424 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.420+0000 7f4f2f3c0700 1 --2- 172.21.15.12:0/3974556568 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f4f0c0354f0 0x7f4f0c0379b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:55.425 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.420+0000 7f4f2f3c0700 1 --2- 172.21.15.12:0/3974556568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4f2806a600 0x7f4f281be2e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:55.425 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.420+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3974556568 >> 172.21.15.12:0/3974556568 conn(0x7f4f28098c00 msgr2=0x7f4f28094fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:55.425 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.420+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3974556568 shutdown_connections 2024-09-13T23:42:55.425 INFO:teuthology.misc.health.smithi012.stderr:2024-09-13T23:42:55.420+0000 7f4f2f3c0700 1 -- 172.21.15.12:0/3974556568 wait complete. 2024-09-13T23:42:55.435 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-09-13T23:42:55.435 INFO:tasks.ceph:Creating RBD pool 2024-09-13T23:42:55.435 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T23:42:55.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.544+0000 7fe038301700 1 Processor -- start 2024-09-13T23:42:55.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.544+0000 7fe038301700 1 -- start start 2024-09-13T23:42:55.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe038301700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe03006f1b0 0x7fe03012f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:55.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe038301700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe030070030 con 0x7fe03006e6c0 2024-09-13T23:42:55.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe038301700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03006fa50 con 0x7fe03006f1b0 2024-09-13T23:42:55.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe03609d700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe03006e6c0 0x7fe03006eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59652/0 (socket says 172.21.15.12:59652) 2024-09-13T23:42:55.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe03609d700 1 -- 172.21.15.12:0/2778095647 learned_addr learned my addr 172.21.15.12:0/2778095647 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:42:55.552 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe03589c700 1 --2- 172.21.15.12:0/2778095647 >> v2:172.21.15.12:3300/0 conn(0x7fe03006f1b0 0x7fe03012f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:55.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe03509b700 1 -- 172.21.15.12:0/2778095647 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550932483 0 0) 0x7fe030070030 con 0x7fe03006e6c0 2024-09-13T23:42:55.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe03509b700 1 -- 172.21.15.12:0/2778095647 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe01c0035d0 con 0x7fe03006e6c0 2024-09-13T23:42:55.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe03589c700 1 -- 172.21.15.12:0/2778095647 >> v1:172.21.15.12:6789/0 conn(0x7fe03006e6c0 legacy=0x7fe03006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:55.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe03509b700 1 -- 172.21.15.12:0/2778095647 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2257726800 0 0) 0x7fe01c0035d0 con 0x7fe03006e6c0 2024-09-13T23:42:55.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe03589c700 1 -- 172.21.15.12:0/2778095647 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe030070470 con 0x7fe03006f1b0 2024-09-13T23:42:55.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe03589c700 1 --2- 172.21.15.12:0/2778095647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe03006f1b0 0x7fe03012f1c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0200097d0 tx=0x7fe020009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181c8a4f930703dd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:55.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe03509b700 1 -- 172.21.15.12:0/2778095647 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe02c0015a0 con 0x7fe03006f1b0 2024-09-13T23:42:55.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe03509b700 1 -- 172.21.15.12:0/2778095647 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe020004030 con 0x7fe03006f1b0 2024-09-13T23:42:55.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe03509b700 1 -- 172.21.15.12:0/2778095647 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0200042d0 con 0x7fe03006f1b0 2024-09-13T23:42:55.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe038301700 1 -- 172.21.15.12:0/2778095647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe03006f1b0 msgr2=0x7fe03012f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:55.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe038301700 1 --2- 172.21.15.12:0/2778095647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe03006f1b0 0x7fe03012f1c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0200097d0 tx=0x7fe020009ae0 comp rx=0 tx=0).stop 2024-09-13T23:42:55.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe038301700 1 -- 172.21.15.12:0/2778095647 shutdown_connections 2024-09-13T23:42:55.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe038301700 1 --2- 172.21.15.12:0/2778095647 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe03006f1b0 0x7fe03012f1c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:55.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe038301700 1 -- 172.21.15.12:0/2778095647 >> 172.21.15.12:0/2778095647 conn(0x7fe0300989f0 msgr2=0x7fe030098e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:55.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe038301700 1 -- 172.21.15.12:0/2778095647 shutdown_connections 2024-09-13T23:42:55.555 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.548+0000 7fe038301700 1 -- 172.21.15.12:0/2778095647 wait complete. 2024-09-13T23:42:55.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe038301700 1 Processor -- start 2024-09-13T23:42:55.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe038301700 1 -- start start 2024-09-13T23:42:55.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe038301700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe03006e6c0 0x7fe030095660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:55.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe038301700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe030128d90 con 0x7fe03006e6c0 2024-09-13T23:42:55.556 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe03609d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe03006e6c0 0x7fe030095660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:55.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe03609d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe03006e6c0 0x7fe030095660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39640/0 (socket says 172.21.15.12:39640) 2024-09-13T23:42:55.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe03609d700 1 -- 172.21.15.12:0/2121486017 learned_addr learned my addr 172.21.15.12:0/2121486017 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:42:55.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe03609d700 1 -- 172.21.15.12:0/2121486017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0200094b0 con 0x7fe03006e6c0 2024-09-13T23:42:55.557 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe03609d700 1 --2- 172.21.15.12:0/2121486017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe03006e6c0 0x7fe030095660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c700 tx=0x7fe02c00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:55.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe026ffd700 1 -- 172.21.15.12:0/2121486017 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe02c007490 con 0x7fe03006e6c0 2024-09-13T23:42:55.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe026ffd700 1 -- 172.21.15.12:0/2121486017 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02c007610 con 0x7fe03006e6c0 2024-09-13T23:42:55.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe026ffd700 1 -- 172.21.15.12:0/2121486017 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe02c004890 con 0x7fe03006e6c0 2024-09-13T23:42:55.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe038301700 1 -- 172.21.15.12:0/2121486017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03006dfe0 con 0x7fe03006e6c0 2024-09-13T23:42:55.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.552+0000 7fe038301700 1 -- 172.21.15.12:0/2121486017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe030127800 con 0x7fe03006e6c0 2024-09-13T23:42:55.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.556+0000 7fe038301700 1 -- 172.21.15.12:0/2121486017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0301280c0 con 0x7fe03006e6c0 2024-09-13T23:42:55.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.556+0000 7fe026ffd700 1 -- 172.21.15.12:0/2121486017 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fe02c004ad0 con 0x7fe03006e6c0 2024-09-13T23:42:55.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.556+0000 7fe026ffd700 1 --2- 172.21.15.12:0/2121486017 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fe0140354a0 0x7fe014037960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:55.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.556+0000 7fe026ffd700 1 -- 172.21.15.12:0/2121486017 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3129+0+0 (secure 0 0 0) 0x7fe02c0471d0 con 0x7fe03006e6c0 2024-09-13T23:42:55.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.556+0000 7fe03589c700 1 --2- 172.21.15.12:0/2121486017 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fe0140354a0 0x7fe014037960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:55.561 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.556+0000 7fe03589c700 1 --2- 172.21.15.12:0/2121486017 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fe0140354a0 0x7fe014037960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe020009b20 tx=0x7fe020008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:55.568 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.564+0000 7fe026ffd700 1 -- 172.21.15.12:0/2121486017 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe02c01a2b0 con 0x7fe03006e6c0 2024-09-13T23:42:55.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:55.736+0000 7fe038301700 1 -- 172.21.15.12:0/2121486017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fe030071c40 con 0x7fe03006e6c0 2024-09-13T23:42:56.528 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.524+0000 7fe026ffd700 1 -- 172.21.15.12:0/2121486017 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) v1 ==== 109+0+0 (secure 0 0 0) 0x7fe02c01bb30 con 0x7fe03006e6c0 2024-09-13T23:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.528+0000 7fe038301700 1 -- 172.21.15.12:0/2121486017 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fe0140354a0 msgr2=0x7fe014037960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.528+0000 7fe038301700 1 --2- 172.21.15.12:0/2121486017 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fe0140354a0 0x7fe014037960 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe020009b20 tx=0x7fe020008030 comp rx=0 tx=0).stop 2024-09-13T23:42:56.533 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.528+0000 7fe038301700 1 -- 172.21.15.12:0/2121486017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe03006e6c0 msgr2=0x7fe030095660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.528+0000 7fe038301700 1 --2- 172.21.15.12:0/2121486017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe03006e6c0 0x7fe030095660 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe02c00c700 tx=0x7fe02c00cac0 comp rx=0 tx=0).stop 2024-09-13T23:42:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.528+0000 7fe038301700 1 -- 172.21.15.12:0/2121486017 shutdown_connections 2024-09-13T23:42:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.528+0000 7fe038301700 1 --2- 172.21.15.12:0/2121486017 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fe0140354a0 0x7fe014037960 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.528+0000 7fe038301700 1 --2- 172.21.15.12:0/2121486017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe03006e6c0 0x7fe030095660 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.528+0000 7fe038301700 1 -- 172.21.15.12:0/2121486017 >> 172.21.15.12:0/2121486017 conn(0x7fe0300989f0 msgr2=0x7fe030097430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.528+0000 7fe038301700 1 -- 172.21.15.12:0/2121486017 shutdown_connections 2024-09-13T23:42:56.534 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.528+0000 7fe038301700 1 -- 172.21.15.12:0/2121486017 wait complete. 2024-09-13T23:42:56.535 INFO:teuthology.orchestra.run.smithi012.stderr:pool 'rbd' created 2024-09-13T23:42:56.550 DEBUG:teuthology.orchestra.run.smithi012:> rbd --cluster ceph pool init rbd 2024-09-13T23:42:56.591 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:42:56.591 INFO:teuthology.orchestra.run.smithi012.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:42:56.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee966e780 1 Processor -- start 2024-09-13T23:42:56.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee966e780 1 -- start start 2024-09-13T23:42:56.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee966e780 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56468e321b90 0x56468e321f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:56.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee966e780 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x56468e1bdca0 con 0x56468e3cdda0 2024-09-13T23:42:56.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee966e780 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x56468e200540 con 0x56468e321b90 2024-09-13T23:42:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee81a5700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56468e321b90 0x56468e321f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee79a4700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x56468e3cdda0 0x56468e3ce1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:59654/0 (socket says 172.21.15.12:59654) 2024-09-13T23:42:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee79a4700 1 -- 172.21.15.12:0/2699429084 learned_addr learned my addr 172.21.15.12:0/2699429084 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:42:56.593 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee81a5700 1 -- 172.21.15.12:0/2699429084 >> v1:172.21.15.12:6789/0 conn(0x56468e3cdda0 legacy=0x56468e3ce1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee81a5700 1 -- 172.21.15.12:0/2699429084 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56468e196bb0 con 0x56468e321b90 2024-09-13T23:42:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee81a5700 1 --2- 172.21.15.12:0/2699429084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56468e321b90 0x56468e321f70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faed000a410 tx=0x7faed000a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb44349948d212c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee71a3700 1 -- 172.21.15.12:0/2699429084 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faed0008460 con 0x56468e321b90 2024-09-13T23:42:56.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee71a3700 1 -- 172.21.15.12:0/2699429084 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed00085c0 con 0x56468e321b90 2024-09-13T23:42:56.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee71a3700 1 -- 172.21.15.12:0/2699429084 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faed00088a0 con 0x56468e321b90 2024-09-13T23:42:56.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee966e780 1 -- 172.21.15.12:0/2699429084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56468e321b90 msgr2=0x56468e321f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:56.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee966e780 1 --2- 172.21.15.12:0/2699429084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56468e321b90 0x56468e321f70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faed000a410 tx=0x7faed000a440 comp rx=0 tx=0).stop 2024-09-13T23:42:56.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.588+0000 7faee966e780 1 -- 172.21.15.12:0/2699429084 shutdown_connections 2024-09-13T23:42:56.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.592+0000 7faee966e780 1 --2- 172.21.15.12:0/2699429084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56468e321b90 0x56468e321f70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:56.595 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.592+0000 7faee966e780 1 -- 172.21.15.12:0/2699429084 >> 172.21.15.12:0/2699429084 conn(0x56468e322aa0 msgr2=0x56468e322eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:56.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.592+0000 7faee966e780 1 -- 172.21.15.12:0/2699429084 shutdown_connections 2024-09-13T23:42:56.596 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.592+0000 7faee966e780 1 -- 172.21.15.12:0/2699429084 wait complete. 2024-09-13T23:42:56.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.592+0000 7faee966e780 1 Processor -- start 2024-09-13T23:42:56.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.592+0000 7faee966e780 1 -- start start 2024-09-13T23:42:56.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.592+0000 7faee966e780 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56468e321b90 0x56468e3bffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:56.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.592+0000 7faee966e780 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468e1fdb10 con 0x56468e321b90 2024-09-13T23:42:56.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.592+0000 7faee81a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56468e321b90 0x56468e3bffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:56.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.592+0000 7faee81a5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56468e321b90 0x56468e3bffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39660/0 (socket says 172.21.15.12:39660) 2024-09-13T23:42:56.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.592+0000 7faee81a5700 1 -- 172.21.15.12:0/1007655509 learned_addr learned my addr 172.21.15.12:0/1007655509 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:42:56.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.596+0000 7faee81a5700 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed0005cc0 con 0x56468e321b90 2024-09-13T23:42:56.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.596+0000 7faee81a5700 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56468e321b90 0x56468e3bffd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faed000b010 tx=0x7faed0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:56.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.596+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faed0008460 con 0x56468e321b90 2024-09-13T23:42:56.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.596+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed000cb60 con 0x56468e321b90 2024-09-13T23:42:56.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.596+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56468e1a3b00 con 0x56468e321b90 2024-09-13T23:42:56.600 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.596+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faed000cd10 con 0x56468e321b90 2024-09-13T23:42:56.601 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.596+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56468e196f40 con 0x56468e321b90 2024-09-13T23:42:56.602 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.596+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7faed00133f0 con 0x56468e321b90 2024-09-13T23:42:56.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.596+0000 7faee499e700 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7faec8033cd0 0x7faec8036190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:56.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.596+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3471+0+0 (secure 0 0 0) 0x7faed0047d10 con 0x56468e321b90 2024-09-13T23:42:56.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.596+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool application enable","pool": "rbd","app": "rbd"} v 0) v1 -- 0x56468e1969a0 con 0x56468e321b90 2024-09-13T23:42:56.603 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.596+0000 7faee79a4700 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7faec8033cd0 0x7faec8036190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:56.604 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:56.600+0000 7faee79a4700 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7faec8033cd0 0x7faec8036190 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faed8000f60 tx=0x7faed80012b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:57.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.532+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "osd pool application enable","pool": "rbd","app": "rbd"}]=0 enabled application 'rbd' on pool 'rbd' v16) v1 ==== 141+0+0 (secure 0 0 0) 0x7faed0023020 con 0x56468e321b90 2024-09-13T23:42:57.539 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.532+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x56468e1bcdd0 con 0x56468e321b90 2024-09-13T23:42:57.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.536+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7faed0006500 con 0x56468e321b90 2024-09-13T23:42:57.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.536+0000 7faee69a2700 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=16}) v3 -- 0x7faecc002ae0 con 0x56468e321b90 2024-09-13T23:42:57.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.536+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 <== mon.0 v2:172.21.15.12:3300/0 8 ==== osd_map(16..16 src has 1..16) v4 ==== 650+0+0 (secure 0 0 0) 0x7faed0026760 con 0x56468e321b90 2024-09-13T23:42:57.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.536+0000 7faee966e780 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56468e3cdda0 0x56468e3c0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:57.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.536+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16) v8 -- 0x56468e3e12d0 con 0x56468e3cdda0 2024-09-13T23:42:57.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.536+0000 7faee9407700 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56468e3cdda0 0x56468e3c0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:57.542 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.536+0000 7faee9407700 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56468e3cdda0 0x56468e3c0f10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:57.550 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.544+0000 7faee9407700 1 -- 172.21.15.12:0/1007655509 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 rbd_trash [create] v16'1 uv1 ondisk = 0) v8 ==== 153+0+0 (crc 0 0 0) 0x7faee000a490 con 0x56468e3cdda0 2024-09-13T23:42:57.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.548+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=33b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16) v8 -- 0x56468e430e10 con 0x56468e3cdda0 2024-09-13T23:42:57.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.548+0000 7faee9407700 1 -- 172.21.15.12:0/1007655509 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(2 rbd_info [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7faee000a490 con 0x56468e3cdda0 2024-09-13T23:42:57.553 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.548+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e16) v8 -- 0x56468e42ff20 con 0x56468e3cdda0 2024-09-13T23:42:57.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.548+0000 7faee9407700 1 -- 172.21.15.12:0/1007655509 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(3 rbd_info [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x7faee000a490 con 0x56468e3cdda0 2024-09-13T23:42:57.554 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:57.548+0000 7faee61a1700 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- pool_op(create unmanaged snap pool 2 tid 4 name v0) v4 -- 0x7faec0001f20 con 0x56468e321b90 2024-09-13T23:42:58.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:58.536+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 <== mon.0 v2:172.21.15.12:3300/0 9 ==== pool_op_reply(tid 4 (0) Success v17) v1 ==== 55+0+0 (secure 0 0 0) 0x7faed0023030 con 0x56468e321b90 2024-09-13T23:42:58.540 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:58.536+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=17}) v3 -- 0x7faec8039770 con 0x56468e321b90 2024-09-13T23:42:58.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:58.536+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 <== mon.0 v2:172.21.15.12:3300/0 10 ==== osd_map(17..17 src has 1..17) v4 ==== 626+0+0 (secure 0 0 0) 0x7faed00273b0 con 0x56468e321b90 2024-09-13T23:42:58.541 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:58.536+0000 7faee69a2700 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8 in=8b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17) v8 -- 0x7faecc005600 con 0x56468e3cdda0 2024-09-13T23:42:58.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:58.540+0000 7faee9407700 1 -- 172.21.15.12:0/1007655509 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(5 rbd_info [create,write 0~8] v17'1 uv1 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x7faee000a490 con 0x56468e3cdda0 2024-09-13T23:42:58.545 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:58.540+0000 7faee61a1700 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- pool_op(delete unmanaged snap pool 2 tid 6 name v17) v4 -- 0x7faec00033a0 con 0x56468e321b90 2024-09-13T23:42:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.544+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 <== mon.0 v2:172.21.15.12:3300/0 11 ==== pool_op_reply(tid 6 (0) Success v18) v1 ==== 43+0+0 (secure 0 0 0) 0x7faed0006320 con 0x56468e321b90 2024-09-13T23:42:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.544+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=18}) v3 -- 0x7faec8039770 con 0x56468e321b90 2024-09-13T23:42:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.544+0000 7faee499e700 1 -- 172.21.15.12:0/1007655509 <== mon.0 v2:172.21.15.12:3300/0 12 ==== osd_map(18..18 src has 1..18) v4 ==== 654+0+0 (secure 0 0 0) 0x7faed0007ce0 con 0x56468e321b90 2024-09-13T23:42:59.551 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.548+0000 7faee69a2700 1 -- 172.21.15.12:0/1007655509 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19 in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18) v8 -- 0x7faecc0059f0 con 0x56468e3cdda0 2024-09-13T23:42:59.558 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.552+0000 7faee9407700 1 -- 172.21.15.12:0/1007655509 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(7 rbd_info [write 0~19] v18'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x7faee000a490 con 0x56468e3cdda0 2024-09-13T23:42:59.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.552+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56468e3cdda0 msgr2=0x56468e3c0f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:59.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.552+0000 7faee966e780 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56468e3cdda0 0x56468e3c0f10 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:59.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.552+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7faec8033cd0 msgr2=0x7faec8036190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:59.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.552+0000 7faee966e780 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7faec8033cd0 0x7faec8036190 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7faed8000f60 tx=0x7faed80012b0 comp rx=0 tx=0).stop 2024-09-13T23:42:59.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.552+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56468e321b90 msgr2=0x56468e3bffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:59.559 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.552+0000 7faee966e780 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56468e321b90 0x56468e3bffd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7faed000b010 tx=0x7faed0002fb0 comp rx=0 tx=0).stop 2024-09-13T23:42:59.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.556+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 shutdown_connections 2024-09-13T23:42:59.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.556+0000 7faee966e780 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56468e3cdda0 0x56468e3c0f10 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:59.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.556+0000 7faee966e780 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7faec8033cd0 0x7faec8036190 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:59.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.556+0000 7faee966e780 1 --2- 172.21.15.12:0/1007655509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56468e321b90 0x56468e3bffd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:59.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.556+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 >> 172.21.15.12:0/1007655509 conn(0x56468e322aa0 msgr2=0x56468e3b9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:59.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.556+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 shutdown_connections 2024-09-13T23:42:59.560 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.556+0000 7faee966e780 1 -- 172.21.15.12:0/1007655509 wait complete. 2024-09-13T23:42:59.566 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T23:42:59.566 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T23:42:59.567 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T23:42:59.567 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T23:42:59.567 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T23:42:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.728+0000 7fbb2cc10700 1 Processor -- start 2024-09-13T23:42:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.728+0000 7fbb2cc10700 1 -- start start 2024-09-13T23:42:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.728+0000 7fbb2cc10700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbb2806f920 0x7fbb2812b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.728+0000 7fbb2cc10700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb2806c230 con 0x7fbb2812ad60 2024-09-13T23:42:59.734 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.728+0000 7fbb2cc10700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb28111d80 con 0x7fbb2806f920 2024-09-13T23:42:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.728+0000 7fbb277fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbb2806f920 0x7fbb2812b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.728+0000 7fbb26ffd700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbb2812ad60 0x7fbb28127570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48682/0 (socket says 172.21.15.12:48682) 2024-09-13T23:42:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.728+0000 7fbb26ffd700 1 -- 172.21.15.12:0/1433125832 learned_addr learned my addr 172.21.15.12:0/1433125832 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:42:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.728+0000 7fbb277fe700 1 -- 172.21.15.12:0/1433125832 >> v1:172.21.15.12:6789/0 conn(0x7fbb2812ad60 legacy=0x7fbb28127570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.728+0000 7fbb277fe700 1 -- 172.21.15.12:0/1433125832 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb28112450 con 0x7fbb2806f920 2024-09-13T23:42:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.728+0000 7fbb267fc700 1 -- 172.21.15.12:0/1433125832 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582056102 0 0) 0x7fbb2806c230 con 0x7fbb2812ad60 2024-09-13T23:42:59.735 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb277fe700 1 --2- 172.21.15.12:0/1433125832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb2806f920 0x7fbb2812b210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb100097d0 tx=0x7fbb10009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f8777d66f98822 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb267fc700 1 -- 172.21.15.12:0/1433125832 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb100041c0 con 0x7fbb2806f920 2024-09-13T23:42:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb267fc700 1 -- 172.21.15.12:0/1433125832 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb10004320 con 0x7fbb2806f920 2024-09-13T23:42:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb267fc700 1 -- 172.21.15.12:0/1433125832 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb100036c0 con 0x7fbb2806f920 2024-09-13T23:42:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1433125832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb2806f920 msgr2=0x7fbb2812b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb2cc10700 1 --2- 172.21.15.12:0/1433125832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb2806f920 0x7fbb2812b210 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbb100097d0 tx=0x7fbb10009ae0 comp rx=0 tx=0).stop 2024-09-13T23:42:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1433125832 shutdown_connections 2024-09-13T23:42:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb2cc10700 1 --2- 172.21.15.12:0/1433125832 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb2806f920 0x7fbb2812b210 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1433125832 >> 172.21.15.12:0/1433125832 conn(0x7fbb280991a0 msgr2=0x7fbb280995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1433125832 shutdown_connections 2024-09-13T23:42:59.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1433125832 wait complete. 2024-09-13T23:42:59.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb2cc10700 1 Processor -- start 2024-09-13T23:42:59.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb2cc10700 1 -- start start 2024-09-13T23:42:59.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb2cc10700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb2806f920 0x7fbb281c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:59.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb2cc10700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb28119850 con 0x7fbb2806f920 2024-09-13T23:42:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb277fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb2806f920 0x7fbb281c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb277fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb2806f920 0x7fbb281c1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39338/0 (socket says 172.21.15.12:39338) 2024-09-13T23:42:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb277fe700 1 -- 172.21.15.12:0/1123508505 learned_addr learned my addr 172.21.15.12:0/1123508505 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:42:59.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb277fe700 1 -- 172.21.15.12:0/1123508505 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100094b0 con 0x7fbb2806f920 2024-09-13T23:42:59.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.732+0000 7fbb277fe700 1 --2- 172.21.15.12:0/1123508505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb2806f920 0x7fbb281c1b70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb100090b0 tx=0x7fbb10004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:59.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.736+0000 7fbb1ffff700 1 -- 172.21.15.12:0/1123508505 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb10006ea0 con 0x7fbb2806f920 2024-09-13T23:42:59.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.736+0000 7fbb1ffff700 1 -- 172.21.15.12:0/1123508505 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb10011760 con 0x7fbb2806f920 2024-09-13T23:42:59.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.736+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1123508505 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb28125240 con 0x7fbb2806f920 2024-09-13T23:42:59.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.736+0000 7fbb1ffff700 1 -- 172.21.15.12:0/1123508505 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb10011a20 con 0x7fbb2806f920 2024-09-13T23:42:59.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.736+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1123508505 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb2811a1a0 con 0x7fbb2806f920 2024-09-13T23:42:59.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.736+0000 7fbb1ffff700 1 -- 172.21.15.12:0/1123508505 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fbb10018050 con 0x7fbb2806f920 2024-09-13T23:42:59.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.736+0000 7fbb1ffff700 1 --2- 172.21.15.12:0/1123508505 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fbb080354f0 0x7fbb080379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:42:59.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.736+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1123508505 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb28119d00 con 0x7fbb2806f920 2024-09-13T23:42:59.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.736+0000 7fbb1ffff700 1 -- 172.21.15.12:0/1123508505 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7fbb10048560 con 0x7fbb2806f920 2024-09-13T23:42:59.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.736+0000 7fbb26ffd700 1 --2- 172.21.15.12:0/1123508505 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fbb080354f0 0x7fbb080379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:42:59.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.736+0000 7fbb26ffd700 1 --2- 172.21.15.12:0/1123508505 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fbb080354f0 0x7fbb080379b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005cb0 tx=0x7fbb180016b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:42:59.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.744+0000 7fbb1ffff700 1 -- 172.21.15.12:0/1123508505 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbb10016070 con 0x7fbb2806f920 2024-09-13T23:42:59.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.956+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1123508505 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fbb281281c0 con 0x7fbb2806f920 2024-09-13T23:42:59.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.960+0000 7fbb1ffff700 1 -- 172.21.15.12:0/1123508505 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+83121 (secure 0 0 0) 0x7fbb1002b030 con 0x7fbb2806f920 2024-09-13T23:42:59.964 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:42:59.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.960+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1123508505 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fbb080354f0 msgr2=0x7fbb080379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:59.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.960+0000 7fbb2cc10700 1 --2- 172.21.15.12:0/1123508505 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fbb080354f0 0x7fbb080379b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbb18005cb0 tx=0x7fbb180016b0 comp rx=0 tx=0).stop 2024-09-13T23:42:59.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.960+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1123508505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb2806f920 msgr2=0x7fbb281c1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:42:59.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.960+0000 7fbb2cc10700 1 --2- 172.21.15.12:0/1123508505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb2806f920 0x7fbb281c1b70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb100090b0 tx=0x7fbb10004770 comp rx=0 tx=0).stop 2024-09-13T23:42:59.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.960+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1123508505 shutdown_connections 2024-09-13T23:42:59.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.960+0000 7fbb2cc10700 1 --2- 172.21.15.12:0/1123508505 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fbb080354f0 0x7fbb080379b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:59.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.960+0000 7fbb2cc10700 1 --2- 172.21.15.12:0/1123508505 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb2806f920 0x7fbb281c1b70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:42:59.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.960+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1123508505 >> 172.21.15.12:0/1123508505 conn(0x7fbb280991a0 msgr2=0x7fbb28129a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:42:59.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.960+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1123508505 shutdown_connections 2024-09-13T23:42:59.967 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:42:59.960+0000 7fbb2cc10700 1 -- 172.21.15.12:0/1123508505 wait complete. 2024-09-13T23:42:59.978 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":1427656188},{"type":"v1","addr":"172.21.15.167:6825","nonce":1427656188}]},"active_addr":"172.21.15.167:6825/1427656188","active_change":"2024-09-13T23:42:45.449966+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":3425542270}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":2973001378}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":314514159}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.167:0","nonce":4287464875}]}]} 2024-09-13T23:42:59.980 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T23:42:59.981 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T23:42:59.981 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T23:43:00.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86ce547700 1 Processor -- start 2024-09-13T23:43:00.093 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86ce547700 1 -- start start 2024-09-13T23:43:00.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86ce547700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f86c812d0e0 0x7f86c812d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86ce547700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c806f700 con 0x7f86c812da00 2024-09-13T23:43:00.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86ce547700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c8071390 con 0x7f86c812d0e0 2024-09-13T23:43:00.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86cd545700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f86c812d0e0 0x7f86c812d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86ccd44700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f86c812da00 0x7f86c806d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48692/0 (socket says 172.21.15.12:48692) 2024-09-13T23:43:00.094 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86ccd44700 1 -- 172.21.15.12:0/3827126226 learned_addr learned my addr 172.21.15.12:0/3827126226 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:00.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86cd545700 1 -- 172.21.15.12:0/3827126226 >> v1:172.21.15.12:6789/0 conn(0x7f86c812da00 legacy=0x7f86c806d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86cd545700 1 -- 172.21.15.12:0/3827126226 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c8071080 con 0x7f86c812d0e0 2024-09-13T23:43:00.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86cd545700 1 --2- 172.21.15.12:0/3827126226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86c812d0e0 0x7f86c812d4c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009800 tx=0x7f86b8009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98031c1c084adb90 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86bffff700 1 -- 172.21.15.12:0/3827126226 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86b8004160 con 0x7f86c812d0e0 2024-09-13T23:43:00.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86bffff700 1 -- 172.21.15.12:0/3827126226 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b80042c0 con 0x7f86c812d0e0 2024-09-13T23:43:00.095 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.088+0000 7f86bffff700 1 -- 172.21.15.12:0/3827126226 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86b8004580 con 0x7f86c812d0e0 2024-09-13T23:43:00.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 -- 172.21.15.12:0/3827126226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86c812d0e0 msgr2=0x7f86c812d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 --2- 172.21.15.12:0/3827126226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86c812d0e0 0x7f86c812d4c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009800 tx=0x7f86b8009b10 comp rx=0 tx=0).stop 2024-09-13T23:43:00.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 -- 172.21.15.12:0/3827126226 shutdown_connections 2024-09-13T23:43:00.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 --2- 172.21.15.12:0/3827126226 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86c812d0e0 0x7f86c812d4c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 -- 172.21.15.12:0/3827126226 >> 172.21.15.12:0/3827126226 conn(0x7f86c809cd60 msgr2=0x7f86c806c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 -- 172.21.15.12:0/3827126226 shutdown_connections 2024-09-13T23:43:00.096 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 -- 172.21.15.12:0/3827126226 wait complete. 2024-09-13T23:43:00.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 Processor -- start 2024-09-13T23:43:00.097 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 -- start start 2024-09-13T23:43:00.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86c812d0e0 0x7f86c81c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c8121530 con 0x7f86c812d0e0 2024-09-13T23:43:00.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86cd545700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86c812d0e0 0x7f86c81c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86cd545700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86c812d0e0 0x7f86c81c16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39364/0 (socket says 172.21.15.12:39364) 2024-09-13T23:43:00.098 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86cd545700 1 -- 172.21.15.12:0/4170476174 learned_addr learned my addr 172.21.15.12:0/4170476174 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:00.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86cd545700 1 -- 172.21.15.12:0/4170476174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b80094b0 con 0x7f86c812d0e0 2024-09-13T23:43:00.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86cd545700 1 --2- 172.21.15.12:0/4170476174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86c812d0e0 0x7f86c81c16b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009b50 tx=0x7f86b8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86bdffb700 1 -- 172.21.15.12:0/4170476174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86b8006ea0 con 0x7f86c812d0e0 2024-09-13T23:43:00.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86bdffb700 1 -- 172.21.15.12:0/4170476174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b801f070 con 0x7f86c812d0e0 2024-09-13T23:43:00.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 -- 172.21.15.12:0/4170476174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c811faf0 con 0x7f86c812d0e0 2024-09-13T23:43:00.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86bdffb700 1 -- 172.21.15.12:0/4170476174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86b8011630 con 0x7f86c812d0e0 2024-09-13T23:43:00.099 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.092+0000 7f86ce547700 1 -- 172.21.15.12:0/4170476174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c8120c90 con 0x7f86c812d0e0 2024-09-13T23:43:00.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.096+0000 7f86bdffb700 1 -- 172.21.15.12:0/4170476174 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f86b8021030 con 0x7f86c812d0e0 2024-09-13T23:43:00.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.096+0000 7f86ce547700 1 -- 172.21.15.12:0/4170476174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c812d4c0 con 0x7f86c812d0e0 2024-09-13T23:43:00.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.096+0000 7f86bdffb700 1 --2- 172.21.15.12:0/4170476174 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f86ac0354f0 0x7f86ac0379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.096+0000 7f86bdffb700 1 -- 172.21.15.12:0/4170476174 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f86b8048e10 con 0x7f86c812d0e0 2024-09-13T23:43:00.100 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.096+0000 7f86ccd44700 1 --2- 172.21.15.12:0/4170476174 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f86ac0354f0 0x7f86ac0379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.101 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.096+0000 7f86ccd44700 1 --2- 172.21.15.12:0/4170476174 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f86ac0354f0 0x7f86ac0379b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f86c4000f60 tx=0x7f86c4007b30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.106 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.100+0000 7f86bdffb700 1 -- 172.21.15.12:0/4170476174 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f86b8010380 con 0x7f86c812d0e0 2024-09-13T23:43:00.276 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.272+0000 7f86ce547700 1 -- 172.21.15.12:0/4170476174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86c806be70 con 0x7f86c812d0e0 2024-09-13T23:43:00.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.272+0000 7f86bdffb700 1 -- 172.21.15.12:0/4170476174 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+9799 (secure 0 0 0) 0x7f86b8019bb0 con 0x7f86c812d0e0 2024-09-13T23:43:00.278 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:43:00.278 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":18,"fsid":"dc64727a-88fa-459b-b2f7-e6853bb706a5","created":"2024-09-13T23:42:40.040576+0000","modified":"2024-09-13T23:42:59.539452+0000","last_up_change":"2024-09-13T23:42:49.482698+0000","last_in_change":"2024-09-13T23:42:41.150993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T23:42:49.531017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T23:42:55.782132+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2f894d3-5bb4-46dd-9be1-bbae3cddb090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6801","nonce":2649399468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6803","nonce":2649399468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6807","nonce":2649399468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6805","nonce":2649399468}]},"public_addr":"172.21.15.12:6801/2649399468","cluster_addr":"172.21.15.12:6803/2649399468","heartbeat_back_addr":"172.21.15.12:6807/2649399468","heartbeat_front_addr":"172.21.15.12:6805/2649399468","state":["exists","up"]},{"osd":1,"uuid":"840a539a-6bbb-4ccd-a32c-7bdf4e7e339c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6801","nonce":3875661733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6803","nonce":3875661733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6807","nonce":3875661733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6805","nonce":3875661733}]},"public_addr":"172.21.15.167:6801/3875661733","cluster_addr":"172.21.15.167:6803/3875661733","heartbeat_back_addr":"172.21.15.167:6807/3875661733","heartbeat_front_addr":"172.21.15.167:6805/3875661733","state":["exists","up"]},{"osd":2,"uuid":"b370fb03-da40-4119-8bc0-6caff41336a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1180369139},{"type":"v1","addr":"172.21.15.167:6809","nonce":1180369139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1180369139},{"type":"v1","addr":"172.21.15.167:6811","nonce":1180369139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1180369139},{"type":"v1","addr":"172.21.15.167:6815","nonce":1180369139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1180369139},{"type":"v1","addr":"172.21.15.167:6813","nonce":1180369139}]},"public_addr":"172.21.15.167:6809/1180369139","cluster_addr":"172.21.15.167:6811/1180369139","heartbeat_back_addr":"172.21.15.167:6815/1180369139","heartbeat_front_addr":"172.21.15.167:6813/1180369139","state":["exists","up"]},{"osd":3,"uuid":"cf2a8b98-93cb-4290-a41d-c833ba4c647c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3688363524},{"type":"v1","addr":"172.21.15.167:6817","nonce":3688363524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3688363524},{"type":"v1","addr":"172.21.15.167:6819","nonce":3688363524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3688363524},{"type":"v1","addr":"172.21.15.167:6823","nonce":3688363524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3688363524},{"type":"v1","addr":"172.21.15.167:6821","nonce":3688363524}]},"public_addr":"172.21.15.167:6817/3688363524","cluster_addr":"172.21.15.167:6819/3688363524","heartbeat_back_addr":"172.21.15.167:6823/3688363524","heartbeat_front_addr":"172.21.15.167:6821/3688363524","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T23:42:46.484652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T23:42:46.146279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T23:42:46.609332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T23:42:46.363369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T23:43:00.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.276+0000 7f86ce547700 1 -- 172.21.15.12:0/4170476174 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f86ac0354f0 msgr2=0x7f86ac0379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.276+0000 7f86ce547700 1 --2- 172.21.15.12:0/4170476174 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f86ac0354f0 0x7f86ac0379b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f86c4000f60 tx=0x7f86c4007b30 comp rx=0 tx=0).stop 2024-09-13T23:43:00.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.276+0000 7f86ce547700 1 -- 172.21.15.12:0/4170476174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86c812d0e0 msgr2=0x7f86c81c16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.276+0000 7f86ce547700 1 --2- 172.21.15.12:0/4170476174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86c812d0e0 0x7f86c81c16b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f86b8009b50 tx=0x7f86b8004770 comp rx=0 tx=0).stop 2024-09-13T23:43:00.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.276+0000 7f86ce547700 1 -- 172.21.15.12:0/4170476174 shutdown_connections 2024-09-13T23:43:00.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.276+0000 7f86ce547700 1 --2- 172.21.15.12:0/4170476174 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f86ac0354f0 0x7f86ac0379b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.276+0000 7f86ce547700 1 --2- 172.21.15.12:0/4170476174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f86c812d0e0 0x7f86c81c16b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.276+0000 7f86ce547700 1 -- 172.21.15.12:0/4170476174 >> 172.21.15.12:0/4170476174 conn(0x7f86c809cd60 msgr2=0x7f86c809ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.276+0000 7f86ce547700 1 -- 172.21.15.12:0/4170476174 shutdown_connections 2024-09-13T23:43:00.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.276+0000 7f86ce547700 1 -- 172.21.15.12:0/4170476174 wait complete. 2024-09-13T23:43:00.293 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T23:43:00.293 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T23:43:00.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.400+0000 7f05b59c8700 1 Processor -- start 2024-09-13T23:43:00.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.400+0000 7f05b59c8700 1 -- start start 2024-09-13T23:43:00.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b59c8700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f05b006f920 0x7f05b012b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b59c8700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b006c230 con 0x7f05b012ad60 2024-09-13T23:43:00.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b59c8700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b0111d80 con 0x7f05b006f920 2024-09-13T23:43:00.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05affff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f05b012ad60 0x7f05b0127570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48704/0 (socket says 172.21.15.12:48704) 2024-09-13T23:43:00.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05affff700 1 -- 172.21.15.12:0/34455915 learned_addr learned my addr 172.21.15.12:0/34455915 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:00.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b49c6700 1 --2- 172.21.15.12:0/34455915 >> v2:172.21.15.12:3300/0 conn(0x7f05b006f920 0x7f05b012b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 -- 172.21.15.12:0/34455915 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941952076 0 0) 0x7f05b006c230 con 0x7f05b012ad60 2024-09-13T23:43:00.408 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 -- 172.21.15.12:0/34455915 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05980035d0 con 0x7f05b012ad60 2024-09-13T23:43:00.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 -- 172.21.15.12:0/34455915 <== mon.0 v1:172.21.15.12:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 917624495 0 0) 0x7f05980035d0 con 0x7f05b012ad60 2024-09-13T23:43:00.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 -- 172.21.15.12:0/34455915 >> v2:172.21.15.12:3300/0 conn(0x7f05b006f920 msgr2=0x7f05b012b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T23:43:00.410 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 --2- 172.21.15.12:0/34455915 >> v2:172.21.15.12:3300/0 conn(0x7f05b006f920 0x7f05b012b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 -- 172.21.15.12:0/34455915 --> v1:172.21.15.12:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b0112450 con 0x7f05b012ad60 2024-09-13T23:43:00.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 -- 172.21.15.12:0/34455915 <== mon.0 v1:172.21.15.12:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 222865856 0 0) 0x7f05a00015a0 con 0x7f05b012ad60 2024-09-13T23:43:00.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 -- 172.21.15.12:0/34455915 >> v1:172.21.15.12:6789/0 conn(0x7f05b012ad60 legacy=0x7f05b0127570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 --2- 172.21.15.12:0/34455915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 0x7f0598003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 -- 172.21.15.12:0/34455915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b0111d80 con 0x7f0598003a20 2024-09-13T23:43:00.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b49c6700 1 --2- 172.21.15.12:0/34455915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 0x7f0598003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b49c6700 1 -- 172.21.15.12:0/34455915 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05980035d0 con 0x7f0598003a20 2024-09-13T23:43:00.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b49c6700 1 --2- 172.21.15.12:0/34455915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 0x7f0598003e20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009af0 tx=0x7f05980058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c57df9a68d479f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 -- 172.21.15.12:0/34455915 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05a400e770 con 0x7f0598003a20 2024-09-13T23:43:00.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 -- 172.21.15.12:0/34455915 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a400e8d0 con 0x7f0598003a20 2024-09-13T23:43:00.411 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05af7fe700 1 -- 172.21.15.12:0/34455915 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05a400eb70 con 0x7f0598003a20 2024-09-13T23:43:00.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b59c8700 1 -- 172.21.15.12:0/34455915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 msgr2=0x7f0598003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b59c8700 1 --2- 172.21.15.12:0/34455915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 0x7f0598003e20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f05a4009af0 tx=0x7f05980058a0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b59c8700 1 -- 172.21.15.12:0/34455915 shutdown_connections 2024-09-13T23:43:00.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b59c8700 1 --2- 172.21.15.12:0/34455915 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 0x7f0598003e20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b59c8700 1 --2- 172.21.15.12:0/34455915 >> v2:172.21.15.12:3300/0 conn(0x7f05b006f920 0x7f05b012b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b59c8700 1 -- 172.21.15.12:0/34455915 >> 172.21.15.12:0/34455915 conn(0x7f05b00991a0 msgr2=0x7f05b00995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b59c8700 1 -- 172.21.15.12:0/34455915 shutdown_connections 2024-09-13T23:43:00.412 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b59c8700 1 -- 172.21.15.12:0/34455915 wait complete. 2024-09-13T23:43:00.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.404+0000 7f05b59c8700 1 Processor -- start 2024-09-13T23:43:00.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05b59c8700 1 -- start start 2024-09-13T23:43:00.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05b59c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 0x7f05b01154f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05b59c8700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b01179f0 con 0x7f0598003a20 2024-09-13T23:43:00.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05b49c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 0x7f05b01154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05b49c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 0x7f05b01154f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39390/0 (socket says 172.21.15.12:39390) 2024-09-13T23:43:00.414 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05b49c6700 1 -- 172.21.15.12:0/2111453890 learned_addr learned my addr 172.21.15.12:0/2111453890 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:00.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05b49c6700 1 -- 172.21.15.12:0/2111453890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b0117290 con 0x7f0598003a20 2024-09-13T23:43:00.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05b49c6700 1 --2- 172.21.15.12:0/2111453890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 0x7f05b01154f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f05b0066f00 tx=0x7f05a40047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05ad7fa700 1 -- 172.21.15.12:0/2111453890 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05a40036c0 con 0x7f0598003a20 2024-09-13T23:43:00.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05b59c8700 1 -- 172.21.15.12:0/2111453890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b0118a60 con 0x7f0598003a20 2024-09-13T23:43:00.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05ad7fa700 1 -- 172.21.15.12:0/2111453890 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a4003840 con 0x7f0598003a20 2024-09-13T23:43:00.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05ad7fa700 1 -- 172.21.15.12:0/2111453890 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05a4003b20 con 0x7f0598003a20 2024-09-13T23:43:00.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05b59c8700 1 -- 172.21.15.12:0/2111453890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b0121b30 con 0x7f0598003a20 2024-09-13T23:43:00.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05ad7fa700 1 -- 172.21.15.12:0/2111453890 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f05a4003d10 con 0x7f0598003a20 2024-09-13T23:43:00.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05ad7fa700 1 --2- 172.21.15.12:0/2111453890 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f059c0354e0 0x7f059c0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05ad7fa700 1 -- 172.21.15.12:0/2111453890 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f05a40490e0 con 0x7f0598003a20 2024-09-13T23:43:00.415 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05affff700 1 --2- 172.21.15.12:0/2111453890 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f059c0354e0 0x7f059c0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05b59c8700 1 -- 172.21.15.12:0/2111453890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b0125240 con 0x7f0598003a20 2024-09-13T23:43:00.416 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.408+0000 7f05affff700 1 --2- 172.21.15.12:0/2111453890 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f059c0354e0 0x7f059c0379a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f05a0000ff0 tx=0x7f05a0001160 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.421 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.416+0000 7f05ad7fa700 1 -- 172.21.15.12:0/2111453890 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f05a4017080 con 0x7f0598003a20 2024-09-13T23:43:00.592 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.588+0000 7f05b59c8700 1 -- 172.21.15.12:0/2111453890 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05b01281c0 con 0x7f0598003a20 2024-09-13T23:43:00.594 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.588+0000 7f05ad7fa700 1 -- 172.21.15.12:0/2111453890 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+9799 (secure 0 0 0) 0x7f05a4026e00 con 0x7f0598003a20 2024-09-13T23:43:00.594 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:43:00.595 INFO:teuthology.orchestra.run.smithi012.stdout:{"epoch":18,"fsid":"dc64727a-88fa-459b-b2f7-e6853bb706a5","created":"2024-09-13T23:42:40.040576+0000","modified":"2024-09-13T23:42:59.539452+0000","last_up_change":"2024-09-13T23:42:49.482698+0000","last_in_change":"2024-09-13T23:42:41.150993+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T23:42:49.531017+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-13T23:42:55.782132+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e2f894d3-5bb4-46dd-9be1-bbae3cddb090","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6800","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6801","nonce":2649399468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6802","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6803","nonce":2649399468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6806","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6807","nonce":2649399468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.12:6804","nonce":2649399468},{"type":"v1","addr":"172.21.15.12:6805","nonce":2649399468}]},"public_addr":"172.21.15.12:6801/2649399468","cluster_addr":"172.21.15.12:6803/2649399468","heartbeat_back_addr":"172.21.15.12:6807/2649399468","heartbeat_front_addr":"172.21.15.12:6805/2649399468","state":["exists","up"]},{"osd":1,"uuid":"840a539a-6bbb-4ccd-a32c-7bdf4e7e339c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6801","nonce":3875661733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6803","nonce":3875661733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6807","nonce":3875661733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":3875661733},{"type":"v1","addr":"172.21.15.167:6805","nonce":3875661733}]},"public_addr":"172.21.15.167:6801/3875661733","cluster_addr":"172.21.15.167:6803/3875661733","heartbeat_back_addr":"172.21.15.167:6807/3875661733","heartbeat_front_addr":"172.21.15.167:6805/3875661733","state":["exists","up"]},{"osd":2,"uuid":"b370fb03-da40-4119-8bc0-6caff41336a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":1180369139},{"type":"v1","addr":"172.21.15.167:6809","nonce":1180369139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":1180369139},{"type":"v1","addr":"172.21.15.167:6811","nonce":1180369139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":1180369139},{"type":"v1","addr":"172.21.15.167:6815","nonce":1180369139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":1180369139},{"type":"v1","addr":"172.21.15.167:6813","nonce":1180369139}]},"public_addr":"172.21.15.167:6809/1180369139","cluster_addr":"172.21.15.167:6811/1180369139","heartbeat_back_addr":"172.21.15.167:6815/1180369139","heartbeat_front_addr":"172.21.15.167:6813/1180369139","state":["exists","up"]},{"osd":3,"uuid":"cf2a8b98-93cb-4290-a41d-c833ba4c647c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":3688363524},{"type":"v1","addr":"172.21.15.167:6817","nonce":3688363524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6818","nonce":3688363524},{"type":"v1","addr":"172.21.15.167:6819","nonce":3688363524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6822","nonce":3688363524},{"type":"v1","addr":"172.21.15.167:6823","nonce":3688363524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":3688363524},{"type":"v1","addr":"172.21.15.167:6821","nonce":3688363524}]},"public_addr":"172.21.15.167:6817/3688363524","cluster_addr":"172.21.15.167:6819/3688363524","heartbeat_back_addr":"172.21.15.167:6823/3688363524","heartbeat_front_addr":"172.21.15.167:6821/3688363524","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T23:42:46.484652+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T23:42:46.146279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T23:42:46.609332+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T23:42:46.363369+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T23:43:00.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.592+0000 7f05b59c8700 1 -- 172.21.15.12:0/2111453890 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f059c0354e0 msgr2=0x7f059c0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.597 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.592+0000 7f05b59c8700 1 --2- 172.21.15.12:0/2111453890 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f059c0354e0 0x7f059c0379a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f05a0000ff0 tx=0x7f05a0001160 comp rx=0 tx=0).stop 2024-09-13T23:43:00.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.592+0000 7f05b59c8700 1 -- 172.21.15.12:0/2111453890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 msgr2=0x7f05b01154f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.592+0000 7f05b59c8700 1 --2- 172.21.15.12:0/2111453890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 0x7f05b01154f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f05b0066f00 tx=0x7f05a40047e0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.592+0000 7f05b59c8700 1 -- 172.21.15.12:0/2111453890 shutdown_connections 2024-09-13T23:43:00.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.592+0000 7f05b59c8700 1 --2- 172.21.15.12:0/2111453890 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f059c0354e0 0x7f059c0379a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.592+0000 7f05b59c8700 1 --2- 172.21.15.12:0/2111453890 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f0598003a20 0x7f05b01154f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.598 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.592+0000 7f05b59c8700 1 -- 172.21.15.12:0/2111453890 >> 172.21.15.12:0/2111453890 conn(0x7f05b00991a0 msgr2=0x7f05b01298e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.592+0000 7f05b59c8700 1 -- 172.21.15.12:0/2111453890 shutdown_connections 2024-09-13T23:43:00.599 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.592+0000 7f05b59c8700 1 -- 172.21.15.12:0/2111453890 wait complete. 2024-09-13T23:43:00.610 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T23:43:00.611 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T23:43:00.612 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T23:43:00.613 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T23:43:00.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c9a4b700 1 Processor -- start 2024-09-13T23:43:00.736 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c9a4b700 1 -- start start 2024-09-13T23:43:00.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c9a4b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19c4095520 0x7f19c4095900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c9a4b700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c4070340 con 0x7f19c4129b40 2024-09-13T23:43:00.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c9a4b700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c40665d0 con 0x7f19c4095520 2024-09-13T23:43:00.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c8a49700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f19c4095520 0x7f19c4095900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c3fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f19c4129b40 0x7f19c4129f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48714/0 (socket says 172.21.15.12:48714) 2024-09-13T23:43:00.737 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c3fff700 1 -- 172.21.15.12:0/23695899 learned_addr learned my addr 172.21.15.12:0/23695899 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:00.738 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c8a49700 1 -- 172.21.15.12:0/23695899 >> v1:172.21.15.12:6789/0 conn(0x7f19c4129b40 legacy=0x7f19c4129f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c8a49700 1 -- 172.21.15.12:0/23695899 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c406ae20 con 0x7f19c4095520 2024-09-13T23:43:00.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c37fe700 1 -- 172.21.15.12:0/23695899 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535213360 0 0) 0x7f19c4070340 con 0x7f19c4129b40 2024-09-13T23:43:00.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c8a49700 1 --2- 172.21.15.12:0/23695899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c4095520 0x7f19c4095900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f19b80097d0 tx=0x7f19b8009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22abab46de453cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c37fe700 1 -- 172.21.15.12:0/23695899 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19b80041c0 con 0x7f19c4095520 2024-09-13T23:43:00.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c37fe700 1 -- 172.21.15.12:0/23695899 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b8004320 con 0x7f19c4095520 2024-09-13T23:43:00.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c37fe700 1 -- 172.21.15.12:0/23695899 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19b80036c0 con 0x7f19c4095520 2024-09-13T23:43:00.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c9a4b700 1 -- 172.21.15.12:0/23695899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c4095520 msgr2=0x7f19c4095900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c9a4b700 1 --2- 172.21.15.12:0/23695899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c4095520 0x7f19c4095900 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f19b80097d0 tx=0x7f19b8009ae0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c9a4b700 1 -- 172.21.15.12:0/23695899 shutdown_connections 2024-09-13T23:43:00.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c9a4b700 1 --2- 172.21.15.12:0/23695899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c4095520 0x7f19c4095900 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c9a4b700 1 -- 172.21.15.12:0/23695899 >> 172.21.15.12:0/23695899 conn(0x7f19c40988a0 msgr2=0x7f19c4098cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.739 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c9a4b700 1 -- 172.21.15.12:0/23695899 shutdown_connections 2024-09-13T23:43:00.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7f19c9a4b700 1 -- 172.21.15.12:0/23695899 wait complete. 2024-09-13T23:43:00.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7fe3bb59e700 1 Processor -- start 2024-09-13T23:43:00.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.732+0000 7fe3bb59e700 1 -- start start 2024-09-13T23:43:00.740 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3bb59e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe3bc1201b0 0x7fe3bc120590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3bb59e700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3bc071110 con 0x7fe3bc071ef0 2024-09-13T23:43:00.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3bb59e700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3bc09af40 con 0x7fe3bc1201b0 2024-09-13T23:43:00.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3ba59c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fe3bc1201b0 0x7fe3bc120590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3b9d9b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fe3bc071ef0 0x7fe3bc06fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48728/0 (socket says 172.21.15.12:48728) 2024-09-13T23:43:00.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3b9d9b700 1 -- 172.21.15.12:0/3176023686 learned_addr learned my addr 172.21.15.12:0/3176023686 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:00.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c9a4b700 1 Processor -- start 2024-09-13T23:43:00.741 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c9a4b700 1 -- start start 2024-09-13T23:43:00.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3ba59c700 1 -- 172.21.15.12:0/3176023686 >> v1:172.21.15.12:6789/0 conn(0x7fe3bc071ef0 legacy=0x7fe3bc06fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3ba59c700 1 -- 172.21.15.12:0/3176023686 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc120ad0 con 0x7fe3bc1201b0 2024-09-13T23:43:00.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3ba59c700 1 --2- 172.21.15.12:0/3176023686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3bc1201b0 0x7fe3bc120590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0097d0 tx=0x7fe3ac009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64724e0606f94f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3b959a700 1 -- 172.21.15.12:0/3176023686 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3ac0041c0 con 0x7fe3bc1201b0 2024-09-13T23:43:00.742 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c9a4b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c4095520 0x7f19c41295c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c9a4b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c4121d00 con 0x7f19c4095520 2024-09-13T23:43:00.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3b959a700 1 -- 172.21.15.12:0/3176023686 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac004320 con 0x7fe3bc1201b0 2024-09-13T23:43:00.743 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3b959a700 1 -- 172.21.15.12:0/3176023686 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3ac0036c0 con 0x7fe3bc1201b0 2024-09-13T23:43:00.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c8a49700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c4095520 0x7f19c41295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c8a49700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c4095520 0x7f19c41295c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39422/0 (socket says 172.21.15.12:39422) 2024-09-13T23:43:00.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c8a49700 1 -- 172.21.15.12:0/1809346767 learned_addr learned my addr 172.21.15.12:0/1809346767 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:00.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c8a49700 1 -- 172.21.15.12:0/1809346767 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b80094b0 con 0x7f19c4095520 2024-09-13T23:43:00.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3bb59e700 1 -- 172.21.15.12:0/3176023686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3bc1201b0 msgr2=0x7fe3bc120590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.744 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3bb59e700 1 --2- 172.21.15.12:0/3176023686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3bc1201b0 0x7fe3bc120590 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0097d0 tx=0x7fe3ac009ae0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c8a49700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c4095520 0x7f19c41295c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f19b80090b0 tx=0x7f19b8004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3bb59e700 1 -- 172.21.15.12:0/3176023686 shutdown_connections 2024-09-13T23:43:00.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3bb59e700 1 --2- 172.21.15.12:0/3176023686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3bc1201b0 0x7fe3bc120590 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3bb59e700 1 -- 172.21.15.12:0/3176023686 >> 172.21.15.12:0/3176023686 conn(0x7fe3bc0991a0 msgr2=0x7fe3bc0995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3bb59e700 1 -- 172.21.15.12:0/3176023686 shutdown_connections 2024-09-13T23:43:00.745 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7fe3bb59e700 1 -- 172.21.15.12:0/3176023686 wait complete. 2024-09-13T23:43:00.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c17fa700 1 -- 172.21.15.12:0/1809346767 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19b8006ea0 con 0x7f19c4095520 2024-09-13T23:43:00.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c17fa700 1 -- 172.21.15.12:0/1809346767 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b8011760 con 0x7f19c4095520 2024-09-13T23:43:00.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c17fa700 1 -- 172.21.15.12:0/1809346767 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19b8011a00 con 0x7f19c4095520 2024-09-13T23:43:00.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c9a4b700 1 -- 172.21.15.12:0/1809346767 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c41224c0 con 0x7f19c4095520 2024-09-13T23:43:00.746 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.736+0000 7f19c9a4b700 1 -- 172.21.15.12:0/1809346767 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c4071c10 con 0x7f19c4095520 2024-09-13T23:43:00.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7f19c17fa700 1 -- 172.21.15.12:0/1809346767 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f19b8011bf0 con 0x7f19c4095520 2024-09-13T23:43:00.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3bb59e700 1 Processor -- start 2024-09-13T23:43:00.747 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3bb59e700 1 -- start start 2024-09-13T23:43:00.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7f19c17fa700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f19a80354f0 0x7f19a80379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7f19c17fa700 1 -- 172.21.15.12:0/1809346767 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f19b8048a00 con 0x7f19c4095520 2024-09-13T23:43:00.748 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7f19c3fff700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f19a80354f0 0x7f19a80379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3bb59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3bc1201b0 0x7fe3bc116a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3bb59e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc118530 con 0x7fe3bc1201b0 2024-09-13T23:43:00.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3bc1201b0 0x7fe3bc116a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3ba59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3bc1201b0 0x7fe3bc116a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39430/0 (socket says 172.21.15.12:39430) 2024-09-13T23:43:00.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3ba59c700 1 -- 172.21.15.12:0/894349910 learned_addr learned my addr 172.21.15.12:0/894349910 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:00.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3ba59c700 1 -- 172.21.15.12:0/894349910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac0094b0 con 0x7fe3bc1201b0 2024-09-13T23:43:00.749 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7f19c3fff700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f19a80354f0 0x7f19a80379b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f19b0000c00 tx=0x7f19b00079c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3ba59c700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3bc1201b0 0x7fe3bc116a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0090b0 tx=0x7fe3ac004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3aaffd700 1 -- 172.21.15.12:0/894349910 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3ac011630 con 0x7fe3bc1201b0 2024-09-13T23:43:00.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3aaffd700 1 -- 172.21.15.12:0/894349910 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac011790 con 0x7fe3bc1201b0 2024-09-13T23:43:00.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3aaffd700 1 -- 172.21.15.12:0/894349910 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe3ac011a30 con 0x7fe3bc1201b0 2024-09-13T23:43:00.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3bb59e700 1 -- 172.21.15.12:0/894349910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc06f680 con 0x7fe3bc1201b0 2024-09-13T23:43:00.750 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7f19c9a4b700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x7f19c4129b00 0x7f19c4120a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7f19c9a4b700 1 -- 172.21.15.12:0/1809346767 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f19c406ff90 con 0x7f19c4129b00 2024-09-13T23:43:00.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3bb59e700 1 -- 172.21.15.12:0/894349910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc124190 con 0x7fe3bc1201b0 2024-09-13T23:43:00.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7f19c924a700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x7f19c4129b00 0x7f19c4120a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.751 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7f19c924a700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x7f19c4129b00 0x7f19c4120a40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.752 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.740+0000 7fe3aaffd700 1 -- 172.21.15.12:0/894349910 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fe3ac018050 con 0x7fe3bc1201b0 2024-09-13T23:43:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fe3bb59e700 1 -- 172.21.15.12:0/894349910 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe3bc0719d0 con 0x7fe3bc1201b0 2024-09-13T23:43:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fe3aaffd700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fe39c0354e0 0x7fe39c0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fe3aaffd700 1 -- 172.21.15.12:0/894349910 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7fe3ac048620 con 0x7fe3bc1201b0 2024-09-13T23:43:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fe3aaffd700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x7fe39c03ac00 0x7fe39c03d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fe3aaffd700 1 -- 172.21.15.12:0/894349910 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe39c03d750 con 0x7fe39c03ac00 2024-09-13T23:43:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fe3aaffd700 1 -- 172.21.15.12:0/894349910 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3ac024950 con 0x7fe3bc1201b0 2024-09-13T23:43:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fe3b9d9b700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fe39c0354e0 0x7fe39c0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fe3bad9d700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x7fe39c03ac00 0x7fe39c03d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fe3b9d9b700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fe39c0354e0 0x7fe39c0379a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0000f60 tx=0x7fe3b0007b30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.753 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fe3bad9d700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x7fe39c03ac00 0x7fe39c03d080 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7f19c17fa700 1 -- 172.21.15.12:0/1809346767 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f19bc011040 con 0x7f19c4129b00 2024-09-13T23:43:00.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fbb5df0b700 1 Processor -- start 2024-09-13T23:43:00.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fbb5df0b700 1 -- start start 2024-09-13T23:43:00.754 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fe3aaffd700 1 -- 172.21.15.12:0/894349910 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fe39c03d750 con 0x7fe39c03ac00 2024-09-13T23:43:00.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fbb5df0b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbb58070700 0x7fbb58132300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fbb5df0b700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5806f8c0 con 0x7fbb5806e480 2024-09-13T23:43:00.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.744+0000 7fbb5df0b700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5806fc90 con 0x7fbb58070700 2024-09-13T23:43:00.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5cf09700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fbb5806e480 0x7fbb5806e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48740/0 (socket says 172.21.15.12:48740) 2024-09-13T23:43:00.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb57fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fbb58070700 0x7fbb58132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.755 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5cf09700 1 -- 172.21.15.12:0/2163080091 learned_addr learned my addr 172.21.15.12:0/2163080091 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:00.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756b59e700 1 Processor -- start 2024-09-13T23:43:00.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756b59e700 1 -- start start 2024-09-13T23:43:00.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb57fff700 1 -- 172.21.15.12:0/2163080091 >> v1:172.21.15.12:6789/0 conn(0x7fbb5806e480 legacy=0x7fbb5806e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.756 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb57fff700 1 -- 172.21.15.12:0/2163080091 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5809af40 con 0x7fbb58070700 2024-09-13T23:43:00.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb57fff700 1 --2- 172.21.15.12:0/2163080091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb58070700 0x7fbb58132300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009d90 tx=0x7fbb5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3302ce212f8e4e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb577fe700 1 -- 172.21.15.12:0/2163080091 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb5000e070 con 0x7fbb58070700 2024-09-13T23:43:00.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb577fe700 1 -- 172.21.15.12:0/2163080091 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5000bc80 con 0x7fbb58070700 2024-09-13T23:43:00.757 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb577fe700 1 -- 172.21.15.12:0/2163080091 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb50003a40 con 0x7fbb58070700 2024-09-13T23:43:00.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756b59e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f756c095900 0x7f756c06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756b59e700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f756c06b0d0 con 0x7f756c06daf0 2024-09-13T23:43:00.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756b59e700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756c067340 con 0x7f756c095900 2024-09-13T23:43:00.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756a59c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f756c095900 0x7f756c06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756a59c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f756c095900 0x7f756c06d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39434/0 (socket says 172.21.15.12:39434) 2024-09-13T23:43:00.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756a59c700 1 -- 172.21.15.12:0/2200879530 learned_addr learned my addr 172.21.15.12:0/2200879530 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:00.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f7569d9b700 1 --1- 172.21.15.12:0/2200879530 >> v1:172.21.15.12:6789/0 conn(0x7f756c06daf0 0x7f756c131b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48750/0 (socket says 172.21.15.12:48750) 2024-09-13T23:43:00.758 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5df0b700 1 -- 172.21.15.12:0/2163080091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb58070700 msgr2=0x7fbb58132300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5df0b700 1 --2- 172.21.15.12:0/2163080091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb58070700 0x7fbb58132300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009d90 tx=0x7fbb5000b330 comp rx=0 tx=0).stop 2024-09-13T23:43:00.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5df0b700 1 -- 172.21.15.12:0/2163080091 shutdown_connections 2024-09-13T23:43:00.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5df0b700 1 --2- 172.21.15.12:0/2163080091 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb58070700 0x7fbb58132300 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5df0b700 1 -- 172.21.15.12:0/2163080091 >> 172.21.15.12:0/2163080091 conn(0x7fbb580991a0 msgr2=0x7fbb580995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756a59c700 1 -- 172.21.15.12:0/2200879530 >> v1:172.21.15.12:6789/0 conn(0x7f756c06daf0 legacy=0x7f756c131b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756a59c700 1 -- 172.21.15.12:0/2200879530 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c06ae20 con 0x7f756c095900 2024-09-13T23:43:00.759 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5df0b700 1 -- 172.21.15.12:0/2163080091 shutdown_connections 2024-09-13T23:43:00.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756a59c700 1 --2- 172.21.15.12:0/2200879530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f756c095900 0x7f756c06d5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f755c009890 tx=0x7f755c009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ca4b48dd008477 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5df0b700 1 -- 172.21.15.12:0/2163080091 wait complete. 2024-09-13T23:43:00.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756959a700 1 -- 172.21.15.12:0/2200879530 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c004160 con 0x7f756c095900 2024-09-13T23:43:00.760 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756959a700 1 -- 172.21.15.12:0/2200879530 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c0042c0 con 0x7f756c095900 2024-09-13T23:43:00.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756959a700 1 -- 172.21.15.12:0/2200879530 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c004580 con 0x7f756c095900 2024-09-13T23:43:00.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5df0b700 1 Processor -- start 2024-09-13T23:43:00.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5df0b700 1 -- start start 2024-09-13T23:43:00.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756b59e700 1 -- 172.21.15.12:0/2200879530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f756c095900 msgr2=0x7f756c06d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.761 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7f756b59e700 1 --2- 172.21.15.12:0/2200879530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f756c095900 0x7f756c06d5b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f755c009890 tx=0x7f755c009ba0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5df0b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5806e480 0x7fbb5811cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5df0b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5811e3a0 con 0x7fbb5806e480 2024-09-13T23:43:00.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5cf09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5806e480 0x7fbb5811cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.762 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5cf09700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5806e480 0x7fbb5811cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39438/0 (socket says 172.21.15.12:39438) 2024-09-13T23:43:00.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5cf09700 1 -- 172.21.15.12:0/3875090313 learned_addr learned my addr 172.21.15.12:0/3875090313 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:00.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.748+0000 7fbb5cf09700 1 -- 172.21.15.12:0/3875090313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50009a70 con 0x7fbb5806e480 2024-09-13T23:43:00.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb5cf09700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5806e480 0x7fbb5811cf80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c680 tx=0x7fbb4800ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756b59e700 1 -- 172.21.15.12:0/2200879530 shutdown_connections 2024-09-13T23:43:00.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756b59e700 1 --2- 172.21.15.12:0/2200879530 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f756c095900 0x7f756c06d5b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.763 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756b59e700 1 -- 172.21.15.12:0/2200879530 >> 172.21.15.12:0/2200879530 conn(0x7f756c0988a0 msgr2=0x7f756c098cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb557fa700 1 -- 172.21.15.12:0/3875090313 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb48005cc0 con 0x7fbb5806e480 2024-09-13T23:43:00.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb557fa700 1 -- 172.21.15.12:0/3875090313 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb48005e20 con 0x7fbb5806e480 2024-09-13T23:43:00.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb557fa700 1 -- 172.21.15.12:0/3875090313 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb48007200 con 0x7fbb5806e480 2024-09-13T23:43:00.764 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756b59e700 1 -- 172.21.15.12:0/2200879530 shutdown_connections 2024-09-13T23:43:00.765 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb5df0b700 1 -- 172.21.15.12:0/3875090313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb58071a30 con 0x7fbb5806e480 2024-09-13T23:43:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb5df0b700 1 -- 172.21.15.12:0/3875090313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5811ea60 con 0x7fbb5806e480 2024-09-13T23:43:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb5df0b700 1 -- 172.21.15.12:0/3875090313 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb581204e0 con 0x7fbb5806e480 2024-09-13T23:43:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb557fa700 1 -- 172.21.15.12:0/3875090313 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fbb48007360 con 0x7fbb5806e480 2024-09-13T23:43:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb557fa700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fbb440354f0 0x7fbb440379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb557fa700 1 -- 172.21.15.12:0/3875090313 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7fbb480469f0 con 0x7fbb5806e480 2024-09-13T23:43:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb557fa700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x7fbb4403ac10 0x7fbb4403d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb557fa700 1 -- 172.21.15.12:0/3875090313 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb4403d760 con 0x7fbb4403ac10 2024-09-13T23:43:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb557fa700 1 -- 172.21.15.12:0/3875090313 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb4804b2a0 con 0x7fbb5806e480 2024-09-13T23:43:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb5d70a700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x7fbb4403ac10 0x7fbb4403d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.766 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb57fff700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fbb440354f0 0x7fbb440379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756b59e700 1 -- 172.21.15.12:0/2200879530 wait complete. 2024-09-13T23:43:00.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756b59e700 1 Processor -- start 2024-09-13T23:43:00.767 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756b59e700 1 -- start start 2024-09-13T23:43:00.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb5d70a700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x7fbb4403ac10 0x7fbb4403d090 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb57fff700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fbb440354f0 0x7fbb440379b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb58070b10 tx=0x7fbb50011000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.768 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7fbb557fa700 1 -- 172.21.15.12:0/3875090313 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fbb4403d760 con 0x7fbb4403ac10 2024-09-13T23:43:00.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756b59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f756c06daf0 0x7f756c1be510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756b59e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c1a7750 con 0x7f756c06daf0 2024-09-13T23:43:00.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f756c06daf0 0x7f756c1be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756a59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f756c06daf0 0x7f756c1be510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39448/0 (socket says 172.21.15.12:39448) 2024-09-13T23:43:00.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756a59c700 1 -- 172.21.15.12:0/3864673406 learned_addr learned my addr 172.21.15.12:0/3864673406 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:00.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756a59c700 1 -- 172.21.15.12:0/3864673406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c009570 con 0x7f756c06daf0 2024-09-13T23:43:00.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.752+0000 7f756a59c700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f756c06daf0 0x7f756c1be510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f755c009be0 tx=0x7f755c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.756+0000 7f755affd700 1 -- 172.21.15.12:0/3864673406 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c003e50 con 0x7f756c06daf0 2024-09-13T23:43:00.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.756+0000 7f755affd700 1 -- 172.21.15.12:0/3864673406 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c00b670 con 0x7f756c06daf0 2024-09-13T23:43:00.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.756+0000 7f756b59e700 1 -- 172.21.15.12:0/3864673406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c1a9940 con 0x7f756c06daf0 2024-09-13T23:43:00.773 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.756+0000 7f755affd700 1 -- 172.21.15.12:0/3864673406 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f755c00b950 con 0x7f756c06daf0 2024-09-13T23:43:00.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.756+0000 7f756b59e700 1 -- 172.21.15.12:0/3864673406 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c1a9c50 con 0x7f756c06daf0 2024-09-13T23:43:00.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.756+0000 7f755affd700 1 -- 172.21.15.12:0/3864673406 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f755c021070 con 0x7f756c06daf0 2024-09-13T23:43:00.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.756+0000 7f755affd700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f754c0354e0 0x7f754c0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.756+0000 7f755affd700 1 -- 172.21.15.12:0/3864673406 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f755c049170 con 0x7f756c06daf0 2024-09-13T23:43:00.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.756+0000 7f7569d9b700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f754c0354e0 0x7f754c0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.756+0000 7f7569d9b700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f754c0354e0 0x7f754c0379a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7564002130 tx=0x7f7564001720 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.760+0000 7f756b59e700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x7f756c095900 0x7f756c1bea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.760+0000 7f756b59e700 1 -- 172.21.15.12:0/3864673406 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f756c06e920 con 0x7f756c095900 2024-09-13T23:43:00.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.760+0000 7f756ad9d700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x7f756c095900 0x7f756c1bea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.760+0000 7f756ad9d700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x7f756c095900 0x7f756c1bea50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.774 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.760+0000 7f755affd700 1 -- 172.21.15.12:0/3864673406 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f7560009850 con 0x7f756c095900 2024-09-13T23:43:00.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 -- 172.21.15.12:0/894349910 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe3bc117d40 con 0x7fe39c03ac00 2024-09-13T23:43:00.775 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3aaffd700 1 -- 172.21.15.12:0/894349910 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe3bc117d40 con 0x7fe39c03ac00 2024-09-13T23:43:00.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 -- 172.21.15.12:0/894349910 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x7fe39c03ac00 msgr2=0x7fe39c03d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x7fe39c03ac00 0x7fe39c03d080 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 -- 172.21.15.12:0/894349910 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fe39c0354e0 msgr2=0x7fe39c0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fe39c0354e0 0x7fe39c0379a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0000f60 tx=0x7fe3b0007b30 comp rx=0 tx=0).stop 2024-09-13T23:43:00.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 -- 172.21.15.12:0/894349910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3bc1201b0 msgr2=0x7fe3bc116a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3bc1201b0 0x7fe3bc116a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0090b0 tx=0x7fe3ac004770 comp rx=0 tx=0).stop 2024-09-13T23:43:00.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 -- 172.21.15.12:0/894349910 shutdown_connections 2024-09-13T23:43:00.776 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x7fe39c03ac00 0x7fe39c03d080 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fe39c0354e0 0x7fe39c0379a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 --2- 172.21.15.12:0/894349910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fe3bc1201b0 0x7fe3bc116a20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 -- 172.21.15.12:0/894349910 >> 172.21.15.12:0/894349910 conn(0x7fe3bc0991a0 msgr2=0x7fe3bc122ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 -- 172.21.15.12:0/894349910 shutdown_connections 2024-09-13T23:43:00.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.764+0000 7fe3bb59e700 1 -- 172.21.15.12:0/894349910 wait complete. 2024-09-13T23:43:00.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.768+0000 7f19c9a4b700 1 -- 172.21.15.12:0/1809346767 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f19c412a660 con 0x7f19c4129b00 2024-09-13T23:43:00.777 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.768+0000 7f19c17fa700 1 -- 172.21.15.12:0/1809346767 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f19bc00acf0 con 0x7f19c4129b00 2024-09-13T23:43:00.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.768+0000 7f19c9a4b700 1 -- 172.21.15.12:0/1809346767 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x7f19c4129b00 msgr2=0x7f19c4120a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.778 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.768+0000 7f19c9a4b700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x7f19c4129b00 0x7f19c4120a40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.768+0000 7f19c9a4b700 1 -- 172.21.15.12:0/1809346767 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f19a80354f0 msgr2=0x7f19a80379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.768+0000 7f19c9a4b700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f19a80354f0 0x7f19a80379b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f19b0000c00 tx=0x7f19b00079c0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.768+0000 7f19c9a4b700 1 -- 172.21.15.12:0/1809346767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c4095520 msgr2=0x7f19c41295c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.768+0000 7f19c9a4b700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c4095520 0x7f19c41295c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f19b80090b0 tx=0x7f19b8004770 comp rx=0 tx=0).stop 2024-09-13T23:43:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.772+0000 7f19c9a4b700 1 -- 172.21.15.12:0/1809346767 shutdown_connections 2024-09-13T23:43:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.772+0000 7f19c9a4b700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x7f19c4129b00 0x7f19c4120a40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.772+0000 7f19c9a4b700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f19a80354f0 0x7f19a80379b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.772+0000 7f19c9a4b700 1 --2- 172.21.15.12:0/1809346767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f19c4095520 0x7f19c41295c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.772+0000 7f19c9a4b700 1 -- 172.21.15.12:0/1809346767 >> 172.21.15.12:0/1809346767 conn(0x7f19c40988a0 msgr2=0x7f19c406ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.772+0000 7f19c9a4b700 1 -- 172.21.15.12:0/1809346767 shutdown_connections 2024-09-13T23:43:00.779 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.772+0000 7f19c9a4b700 1 -- 172.21.15.12:0/1809346767 wait complete. 2024-09-13T23:43:00.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.772+0000 7fbb5df0b700 1 -- 172.21.15.12:0/3875090313 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbb58120950 con 0x7fbb4403ac10 2024-09-13T23:43:00.780 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.772+0000 7fbb557fa700 1 -- 172.21.15.12:0/3875090313 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbb58120950 con 0x7fbb4403ac10 2024-09-13T23:43:00.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 -- 172.21.15.12:0/3875090313 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x7fbb4403ac10 msgr2=0x7fbb4403d090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x7fbb4403ac10 0x7fbb4403d090 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 -- 172.21.15.12:0/3875090313 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fbb440354f0 msgr2=0x7fbb440379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fbb440354f0 0x7fbb440379b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb58070b10 tx=0x7fbb50011000 comp rx=0 tx=0).stop 2024-09-13T23:43:00.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 -- 172.21.15.12:0/3875090313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5806e480 msgr2=0x7fbb5811cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5806e480 0x7fbb5811cf80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c680 tx=0x7fbb4800ca40 comp rx=0 tx=0).stop 2024-09-13T23:43:00.781 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 -- 172.21.15.12:0/3875090313 shutdown_connections 2024-09-13T23:43:00.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x7fbb4403ac10 0x7fbb4403d090 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fbb440354f0 0x7fbb440379b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 --2- 172.21.15.12:0/3875090313 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fbb5806e480 0x7fbb5811cf80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 -- 172.21.15.12:0/3875090313 >> 172.21.15.12:0/3875090313 conn(0x7fbb580991a0 msgr2=0x7fbb581b50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 -- 172.21.15.12:0/3875090313 shutdown_connections 2024-09-13T23:43:00.782 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.776+0000 7fbb5df0b700 1 -- 172.21.15.12:0/3875090313 wait complete. 2024-09-13T23:43:00.783 INFO:teuthology.orchestra.run.smithi012.stdout:47244640262 2024-09-13T23:43:00.783 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T23:43:00.785 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 -- 172.21.15.12:0/3864673406 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f756c06d5b0 con 0x7f756c095900 2024-09-13T23:43:00.786 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f755affd700 1 -- 172.21.15.12:0/3864673406 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f756c06d5b0 con 0x7f756c095900 2024-09-13T23:43:00.787 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 -- 172.21.15.12:0/3864673406 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x7f756c095900 msgr2=0x7f756c1bea50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x7f756c095900 0x7f756c1bea50 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 -- 172.21.15.12:0/3864673406 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f754c0354e0 msgr2=0x7f754c0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f754c0354e0 0x7f754c0379a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7564002130 tx=0x7f7564001720 comp rx=0 tx=0).stop 2024-09-13T23:43:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 -- 172.21.15.12:0/3864673406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f756c06daf0 msgr2=0x7f756c1be510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f756c06daf0 0x7f756c1be510 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f755c009be0 tx=0x7f755c004770 comp rx=0 tx=0).stop 2024-09-13T23:43:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 -- 172.21.15.12:0/3864673406 shutdown_connections 2024-09-13T23:43:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x7f756c095900 0x7f756c1bea50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f754c0354e0 0x7f754c0379a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 --2- 172.21.15.12:0/3864673406 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f756c06daf0 0x7f756c1be510 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 -- 172.21.15.12:0/3864673406 >> 172.21.15.12:0/3864673406 conn(0x7f756c0988a0 msgr2=0x7f756c08f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.788 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.780+0000 7f756b59e700 1 -- 172.21.15.12:0/3864673406 shutdown_connections 2024-09-13T23:43:00.789 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.784+0000 7f756b59e700 1 -- 172.21.15.12:0/3864673406 wait complete. 2024-09-13T23:43:00.789 INFO:teuthology.orchestra.run.smithi012.stdout:42949672967 2024-09-13T23:43:00.790 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T23:43:00.794 INFO:teuthology.orchestra.run.smithi012.stdout:42949672966 2024-09-13T23:43:00.795 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T23:43:00.802 INFO:teuthology.orchestra.run.smithi012.stdout:47244640263 2024-09-13T23:43:00.803 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T23:43:00.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f862c83d700 1 Processor -- start 2024-09-13T23:43:00.881 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f862c83d700 1 -- start start 2024-09-13T23:43:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f862c83d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f862806e2a0 0x7f862806e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f862c83d700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862806ff60 con 0x7f862806ebc0 2024-09-13T23:43:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f862c83d700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8628071490 con 0x7f862806e2a0 2024-09-13T23:43:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f86277fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f862806e2a0 0x7f862806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f8626ffd700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f862806ebc0 0x7f8628129b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48766/0 (socket says 172.21.15.12:48766) 2024-09-13T23:43:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f8626ffd700 1 -- 172.21.15.12:0/3641083717 learned_addr learned my addr 172.21.15.12:0/3641083717 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:00.882 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f86277fe700 1 -- 172.21.15.12:0/3641083717 >> v1:172.21.15.12:6789/0 conn(0x7f862806ebc0 legacy=0x7f8628129b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f86277fe700 1 -- 172.21.15.12:0/3641083717 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8628099c90 con 0x7f862806e2a0 2024-09-13T23:43:00.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f86267fc700 1 -- 172.21.15.12:0/3641083717 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395970532 0 0) 0x7f862806ff60 con 0x7f862806ebc0 2024-09-13T23:43:00.883 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f86277fe700 1 --2- 172.21.15.12:0/3641083717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f862806e2a0 0x7f862806e680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f861c0097d0 tx=0x7f861c009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc711ffa5c50866 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f86267fc700 1 -- 172.21.15.12:0/3641083717 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f861c0041c0 con 0x7f862806e2a0 2024-09-13T23:43:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f86267fc700 1 -- 172.21.15.12:0/3641083717 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c004320 con 0x7f862806e2a0 2024-09-13T23:43:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.876+0000 7f86267fc700 1 -- 172.21.15.12:0/3641083717 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f861c0036c0 con 0x7f862806e2a0 2024-09-13T23:43:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 -- 172.21.15.12:0/3641083717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f862806e2a0 msgr2=0x7f862806e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 --2- 172.21.15.12:0/3641083717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f862806e2a0 0x7f862806e680 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f861c0097d0 tx=0x7f861c009ae0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 -- 172.21.15.12:0/3641083717 shutdown_connections 2024-09-13T23:43:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 --2- 172.21.15.12:0/3641083717 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f862806e2a0 0x7f862806e680 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 -- 172.21.15.12:0/3641083717 >> 172.21.15.12:0/3641083717 conn(0x7f862809b0f0 msgr2=0x7f862809b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 -- 172.21.15.12:0/3641083717 shutdown_connections 2024-09-13T23:43:00.884 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 -- 172.21.15.12:0/3641083717 wait complete. 2024-09-13T23:43:00.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 Processor -- start 2024-09-13T23:43:00.885 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 -- start start 2024-09-13T23:43:00.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f862806e2a0 0x7f86281147e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86281174b0 con 0x7f862806e2a0 2024-09-13T23:43:00.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f86277fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f862806e2a0 0x7f86281147e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f86277fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f862806e2a0 0x7f86281147e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39472/0 (socket says 172.21.15.12:39472) 2024-09-13T23:43:00.886 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f86277fe700 1 -- 172.21.15.12:0/1169089394 learned_addr learned my addr 172.21.15.12:0/1169089394 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:00.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f86277fe700 1 -- 172.21.15.12:0/1169089394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c0094b0 con 0x7f862806e2a0 2024-09-13T23:43:00.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f86277fe700 1 --2- 172.21.15.12:0/1169089394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f862806e2a0 0x7f86281147e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f861c014040 tx=0x7f861c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f860ffff700 1 -- 172.21.15.12:0/1169089394 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f861c011630 con 0x7f862806e2a0 2024-09-13T23:43:00.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f860ffff700 1 -- 172.21.15.12:0/1169089394 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c011790 con 0x7f862806e2a0 2024-09-13T23:43:00.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f860ffff700 1 -- 172.21.15.12:0/1169089394 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f861c011a30 con 0x7f862806e2a0 2024-09-13T23:43:00.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 -- 172.21.15.12:0/1169089394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8628116310 con 0x7f862806e2a0 2024-09-13T23:43:00.887 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.880+0000 7f862c83d700 1 -- 172.21.15.12:0/1169089394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8628116ff0 con 0x7f862806e2a0 2024-09-13T23:43:00.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.884+0000 7f860ffff700 1 -- 172.21.15.12:0/1169089394 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f861c018050 con 0x7f862806e2a0 2024-09-13T23:43:00.888 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.884+0000 7f862c83d700 1 -- 172.21.15.12:0/1169089394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862806e680 con 0x7f862806e2a0 2024-09-13T23:43:00.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.884+0000 7f860ffff700 1 --2- 172.21.15.12:0/1169089394 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f86140354f0 0x7f86140379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.884+0000 7f860ffff700 1 -- 172.21.15.12:0/1169089394 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f861c048620 con 0x7f862806e2a0 2024-09-13T23:43:00.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.884+0000 7f8626ffd700 1 --2- 172.21.15.12:0/1169089394 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f86140354f0 0x7f86140379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.889 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.884+0000 7f8626ffd700 1 --2- 172.21.15.12:0/1169089394 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f86140354f0 0x7f86140379b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8618000c00 tx=0x7f86180079c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.896 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.892+0000 7f860ffff700 1 -- 172.21.15.12:0/1169089394 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f861c024950 con 0x7f862806e2a0 2024-09-13T23:43:00.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.908+0000 7feb78663700 1 Processor -- start 2024-09-13T23:43:00.913 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.908+0000 7feb78663700 1 -- start start 2024-09-13T23:43:00.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.908+0000 7feb78663700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb7012c590 0x7feb7012e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.908+0000 7feb78663700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb70070440 con 0x7feb70129490 2024-09-13T23:43:00.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.908+0000 7feb78663700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb70071c90 con 0x7feb7012c590 2024-09-13T23:43:00.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.908+0000 7feb75bfe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7feb7012c590 0x7feb7012e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.908+0000 7feb763ff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7feb70129490 0x7feb70129870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48768/0 (socket says 172.21.15.12:48768) 2024-09-13T23:43:00.914 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.908+0000 7feb763ff700 1 -- 172.21.15.12:0/1898669675 learned_addr learned my addr 172.21.15.12:0/1898669675 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:00.915 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.908+0000 7feb75bfe700 1 -- 172.21.15.12:0/1898669675 >> v1:172.21.15.12:6789/0 conn(0x7feb70129490 legacy=0x7feb70129870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.908+0000 7feb75bfe700 1 -- 172.21.15.12:0/1898669675 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb70070760 con 0x7feb7012c590 2024-09-13T23:43:00.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.908+0000 7feb75bfe700 1 --2- 172.21.15.12:0/1898669675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb7012c590 0x7feb7012e980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feb6800bfa0 tx=0x7feb6800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824a5240b1951955 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb753fd700 1 -- 172.21.15.12:0/1898669675 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb68010070 con 0x7feb7012c590 2024-09-13T23:43:00.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb753fd700 1 -- 172.21.15.12:0/1898669675 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb68003b70 con 0x7feb7012c590 2024-09-13T23:43:00.916 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb753fd700 1 -- 172.21.15.12:0/1898669675 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb68003e30 con 0x7feb7012c590 2024-09-13T23:43:00.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb78663700 1 -- 172.21.15.12:0/1898669675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb7012c590 msgr2=0x7feb7012e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb78663700 1 --2- 172.21.15.12:0/1898669675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb7012c590 0x7feb7012e980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7feb6800bfa0 tx=0x7feb6800d330 comp rx=0 tx=0).stop 2024-09-13T23:43:00.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb78663700 1 -- 172.21.15.12:0/1898669675 shutdown_connections 2024-09-13T23:43:00.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb78663700 1 --2- 172.21.15.12:0/1898669675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb7012c590 0x7feb7012e980 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb78663700 1 -- 172.21.15.12:0/1898669675 >> 172.21.15.12:0/1898669675 conn(0x7feb70099420 msgr2=0x7feb70099830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb78663700 1 -- 172.21.15.12:0/1898669675 shutdown_connections 2024-09-13T23:43:00.917 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb78663700 1 -- 172.21.15.12:0/1898669675 wait complete. 2024-09-13T23:43:00.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb78663700 1 Processor -- start 2024-09-13T23:43:00.918 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb78663700 1 -- start start 2024-09-13T23:43:00.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb78663700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb70129490 0x7feb701c19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb78663700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb7011d500 con 0x7feb70129490 2024-09-13T23:43:00.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb763ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb70129490 0x7feb701c19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb763ff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb70129490 0x7feb701c19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39490/0 (socket says 172.21.15.12:39490) 2024-09-13T23:43:00.919 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.912+0000 7feb763ff700 1 -- 172.21.15.12:0/19549361 learned_addr learned my addr 172.21.15.12:0/19549361 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:00.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.916+0000 7feb763ff700 1 -- 172.21.15.12:0/19549361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb6800bc80 con 0x7feb70129490 2024-09-13T23:43:00.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.916+0000 7feb763ff700 1 --2- 172.21.15.12:0/19549361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb70129490 0x7feb701c19f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feb6000c480 tx=0x7feb6000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.916+0000 7feb66ffd700 1 -- 172.21.15.12:0/19549361 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb60005cc0 con 0x7feb70129490 2024-09-13T23:43:00.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.916+0000 7feb66ffd700 1 -- 172.21.15.12:0/19549361 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb60005e20 con 0x7feb70129490 2024-09-13T23:43:00.920 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.916+0000 7feb78663700 1 -- 172.21.15.12:0/19549361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb7011e180 con 0x7feb70129490 2024-09-13T23:43:00.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.916+0000 7feb66ffd700 1 -- 172.21.15.12:0/19549361 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb600049c0 con 0x7feb70129490 2024-09-13T23:43:00.921 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.916+0000 7feb78663700 1 -- 172.21.15.12:0/19549361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb70072200 con 0x7feb70129490 2024-09-13T23:43:00.922 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.916+0000 7feb78663700 1 -- 172.21.15.12:0/19549361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb7006e0d0 con 0x7feb70129490 2024-09-13T23:43:00.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.916+0000 7feb66ffd700 1 -- 172.21.15.12:0/19549361 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7feb60004c30 con 0x7feb70129490 2024-09-13T23:43:00.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.920+0000 7feb66ffd700 1 --2- 172.21.15.12:0/19549361 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7feb5c0354f0 0x7feb5c0379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.923 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.920+0000 7feb66ffd700 1 -- 172.21.15.12:0/19549361 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7feb600476f0 con 0x7feb70129490 2024-09-13T23:43:00.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.920+0000 7feb75bfe700 1 --2- 172.21.15.12:0/19549361 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7feb5c0354f0 0x7feb5c0379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.920+0000 7feb75bfe700 1 --2- 172.21.15.12:0/19549361 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7feb5c0354f0 0x7feb5c0379b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb6800d370 tx=0x7feb68012000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.924+0000 7feb66ffd700 1 -- 172.21.15.12:0/19549361 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7feb600070d0 con 0x7feb70129490 2024-09-13T23:43:00.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49db59e700 1 Processor -- start 2024-09-13T23:43:00.951 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49db59e700 1 -- start start 2024-09-13T23:43:00.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49db59e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f49dc12d0e0 0x7f49dc12d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49db59e700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49dc06f700 con 0x7f49dc12da00 2024-09-13T23:43:00.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49db59e700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49dc071390 con 0x7f49dc12d0e0 2024-09-13T23:43:00.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49da59c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f49dc12d0e0 0x7f49dc12d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49d3fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f49dc12da00 0x7f49dc06d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48772/0 (socket says 172.21.15.12:48772) 2024-09-13T23:43:00.953 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49d3fff700 1 -- 172.21.15.12:0/3109949018 learned_addr learned my addr 172.21.15.12:0/3109949018 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:00.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49da59c700 1 -- 172.21.15.12:0/3109949018 >> v1:172.21.15.12:6789/0 conn(0x7f49dc12da00 legacy=0x7f49dc06d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49da59c700 1 -- 172.21.15.12:0/3109949018 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49dc071080 con 0x7f49dc12d0e0 2024-09-13T23:43:00.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49d9d9b700 1 -- 172.21.15.12:0/3109949018 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877917195 0 0) 0x7f49dc06f700 con 0x7f49dc12da00 2024-09-13T23:43:00.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49da59c700 1 --2- 172.21.15.12:0/3109949018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49dc12d0e0 0x7f49dc12d4c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009800 tx=0x7f49c4009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c7b5757b974830 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49d9d9b700 1 -- 172.21.15.12:0/3109949018 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49c4004160 con 0x7f49dc12d0e0 2024-09-13T23:43:00.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49d9d9b700 1 -- 172.21.15.12:0/3109949018 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c40042c0 con 0x7f49dc12d0e0 2024-09-13T23:43:00.954 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49d9d9b700 1 -- 172.21.15.12:0/3109949018 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49c4004580 con 0x7f49dc12d0e0 2024-09-13T23:43:00.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49db59e700 1 -- 172.21.15.12:0/3109949018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49dc12d0e0 msgr2=0x7f49dc12d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49db59e700 1 --2- 172.21.15.12:0/3109949018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49dc12d0e0 0x7f49dc12d4c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009800 tx=0x7f49c4009b10 comp rx=0 tx=0).stop 2024-09-13T23:43:00.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49db59e700 1 -- 172.21.15.12:0/3109949018 shutdown_connections 2024-09-13T23:43:00.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49db59e700 1 --2- 172.21.15.12:0/3109949018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49dc12d0e0 0x7f49dc12d4c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49db59e700 1 -- 172.21.15.12:0/3109949018 >> 172.21.15.12:0/3109949018 conn(0x7f49dc09cd60 msgr2=0x7f49dc06c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49db59e700 1 -- 172.21.15.12:0/3109949018 shutdown_connections 2024-09-13T23:43:00.955 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.948+0000 7f49db59e700 1 -- 172.21.15.12:0/3109949018 wait complete. 2024-09-13T23:43:00.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49db59e700 1 Processor -- start 2024-09-13T23:43:00.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49db59e700 1 -- start start 2024-09-13T23:43:00.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49db59e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49dc12d0e0 0x7f49dc11ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49db59e700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49dc11c600 con 0x7f49dc12d0e0 2024-09-13T23:43:00.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49da59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49dc12d0e0 0x7f49dc11ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49da59c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49dc12d0e0 0x7f49dc11ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39518/0 (socket says 172.21.15.12:39518) 2024-09-13T23:43:00.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49da59c700 1 -- 172.21.15.12:0/1604200480 learned_addr learned my addr 172.21.15.12:0/1604200480 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:00.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49da59c700 1 -- 172.21.15.12:0/1604200480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40094b0 con 0x7f49dc12d0e0 2024-09-13T23:43:00.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49da59c700 1 --2- 172.21.15.12:0/1604200480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49dc12d0e0 0x7f49dc11ad30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f49c40090b0 tx=0x7f49c4004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49d2ffd700 1 -- 172.21.15.12:0/1604200480 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49c4006ea0 con 0x7f49dc12d0e0 2024-09-13T23:43:00.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49d2ffd700 1 -- 172.21.15.12:0/1604200480 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c401f070 con 0x7f49dc12d0e0 2024-09-13T23:43:00.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49db59e700 1 -- 172.21.15.12:0/1604200480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49dc11cb80 con 0x7f49dc12d0e0 2024-09-13T23:43:00.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49d2ffd700 1 -- 172.21.15.12:0/1604200480 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49c4011630 con 0x7f49dc12d0e0 2024-09-13T23:43:00.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49db59e700 1 -- 172.21.15.12:0/1604200480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49dc11e350 con 0x7f49dc12d0e0 2024-09-13T23:43:00.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49d2ffd700 1 -- 172.21.15.12:0/1604200480 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f49c4021030 con 0x7f49dc12d0e0 2024-09-13T23:43:00.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49d2ffd700 1 --2- 172.21.15.12:0/1604200480 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f49bc035540 0x7f49bc037a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49db59e700 1 -- 172.21.15.12:0/1604200480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49dc12d4c0 con 0x7f49dc12d0e0 2024-09-13T23:43:00.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.952+0000 7f49d2ffd700 1 -- 172.21.15.12:0/1604200480 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f49c40492a0 con 0x7f49dc12d0e0 2024-09-13T23:43:00.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.956+0000 7f49d3fff700 1 --2- 172.21.15.12:0/1604200480 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f49bc035540 0x7f49bc037a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.956+0000 7f49d3fff700 1 --2- 172.21.15.12:0/1604200480 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f49bc035540 0x7f49bc037a00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f49cc000c00 tx=0x7f49cc007a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.960+0000 7f49d2ffd700 1 -- 172.21.15.12:0/1604200480 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f49c4010de0 con 0x7f49dc12d0e0 2024-09-13T23:43:00.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.964+0000 7fd2d159c700 1 Processor -- start 2024-09-13T23:43:00.971 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.964+0000 7fd2d159c700 1 -- start start 2024-09-13T23:43:00.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd2cc070700 0x7fd2cc132300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2cc06f8c0 con 0x7fd2cc06e480 2024-09-13T23:43:00.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2cc06fc90 con 0x7fd2cc070700 2024-09-13T23:43:00.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2cb7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd2cc070700 0x7fd2cc132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2cbfff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fd2cc06e480 0x7fd2cc06e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48776/0 (socket says 172.21.15.12:48776) 2024-09-13T23:43:00.972 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2cbfff700 1 -- 172.21.15.12:0/2649335113 learned_addr learned my addr 172.21.15.12:0/2649335113 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:00.973 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2cb7fe700 1 -- 172.21.15.12:0/2649335113 >> v1:172.21.15.12:6789/0 conn(0x7fd2cc06e480 legacy=0x7fd2cc06e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2cb7fe700 1 -- 172.21.15.12:0/2649335113 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc09af40 con 0x7fd2cc070700 2024-09-13T23:43:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2cb7fe700 1 --2- 172.21.15.12:0/2649335113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2cc070700 0x7fd2cc132300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097d0 tx=0x7fd2bc009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38d1ecabd8fdf1c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2caffd700 1 -- 172.21.15.12:0/2649335113 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2bc0041c0 con 0x7fd2cc070700 2024-09-13T23:43:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2caffd700 1 -- 172.21.15.12:0/2649335113 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc004320 con 0x7fd2cc070700 2024-09-13T23:43:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2caffd700 1 -- 172.21.15.12:0/2649335113 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2bc0036c0 con 0x7fd2cc070700 2024-09-13T23:43:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 -- 172.21.15.12:0/2649335113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2cc070700 msgr2=0x7fd2cc132300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 --2- 172.21.15.12:0/2649335113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2cc070700 0x7fd2cc132300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0097d0 tx=0x7fd2bc009ae0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 -- 172.21.15.12:0/2649335113 shutdown_connections 2024-09-13T23:43:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 --2- 172.21.15.12:0/2649335113 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2cc070700 0x7fd2cc132300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 -- 172.21.15.12:0/2649335113 >> 172.21.15.12:0/2649335113 conn(0x7fd2cc0991a0 msgr2=0x7fd2cc0995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 -- 172.21.15.12:0/2649335113 shutdown_connections 2024-09-13T23:43:00.974 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 -- 172.21.15.12:0/2649335113 wait complete. 2024-09-13T23:43:00.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 Processor -- start 2024-09-13T23:43:00.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 -- start start 2024-09-13T23:43:00.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2cc06e480 0x7fd2cc11d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.968+0000 7fd2d159c700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc1205f0 con 0x7fd2cc06e480 2024-09-13T23:43:00.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2cbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2cc06e480 0x7fd2cc11d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2cbfff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2cc06e480 0x7fd2cc11d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39536/0 (socket says 172.21.15.12:39536) 2024-09-13T23:43:00.976 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2cbfff700 1 -- 172.21.15.12:0/2469110386 learned_addr learned my addr 172.21.15.12:0/2469110386 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:00.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2cbfff700 1 -- 172.21.15.12:0/2469110386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc0094b0 con 0x7fd2cc06e480 2024-09-13T23:43:00.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2cbfff700 1 --2- 172.21.15.12:0/2469110386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2cc06e480 0x7fd2cc11d3d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400c680 tx=0x7fd2b400ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2c8ff9700 1 -- 172.21.15.12:0/2469110386 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2b4005cc0 con 0x7fd2cc06e480 2024-09-13T23:43:00.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2c8ff9700 1 -- 172.21.15.12:0/2469110386 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b4005e20 con 0x7fd2cc06e480 2024-09-13T23:43:00.977 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2d159c700 1 -- 172.21.15.12:0/2469110386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc11ebb0 con 0x7fd2cc06e480 2024-09-13T23:43:00.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2c8ff9700 1 -- 172.21.15.12:0/2469110386 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2b4007200 con 0x7fd2cc06e480 2024-09-13T23:43:00.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2d159c700 1 -- 172.21.15.12:0/2469110386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc11fd50 con 0x7fd2cc06e480 2024-09-13T23:43:00.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2d159c700 1 -- 172.21.15.12:0/2469110386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc06e860 con 0x7fd2cc06e480 2024-09-13T23:43:00.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2c8ff9700 1 -- 172.21.15.12:0/2469110386 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fd2b4007360 con 0x7fd2cc06e480 2024-09-13T23:43:00.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2c8ff9700 1 --2- 172.21.15.12:0/2469110386 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fd2ac0354f0 0x7fd2ac0379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:00.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2c8ff9700 1 -- 172.21.15.12:0/2469110386 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7fd2b4046b50 con 0x7fd2cc06e480 2024-09-13T23:43:00.978 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2cb7fe700 1 --2- 172.21.15.12:0/2469110386 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fd2ac0354f0 0x7fd2ac0379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:00.979 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.972+0000 7fd2cb7fe700 1 --2- 172.21.15.12:0/2469110386 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fd2ac0354f0 0x7fd2ac0379b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc070b10 tx=0x7fd2bc010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:00.986 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:00.980+0000 7fd2c8ff9700 1 -- 172.21.15.12:0/2469110386 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd2b4012400 con 0x7fd2cc06e480 2024-09-13T23:43:01.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.068+0000 7f862c83d700 1 -- 172.21.15.12:0/1169089394 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f862806b7b0 con 0x7f862806e2a0 2024-09-13T23:43:01.075 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.068+0000 7f860ffff700 1 -- 172.21.15.12:0/1169089394 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f861c02b020 con 0x7f862806e2a0 2024-09-13T23:43:01.075 INFO:teuthology.orchestra.run.smithi012.stdout:47244640260 2024-09-13T23:43:01.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.072+0000 7f862c83d700 1 -- 172.21.15.12:0/1169089394 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f86140354f0 msgr2=0x7f86140379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:01.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.072+0000 7f862c83d700 1 --2- 172.21.15.12:0/1169089394 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f86140354f0 0x7f86140379b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8618000c00 tx=0x7f86180079c0 comp rx=0 tx=0).stop 2024-09-13T23:43:01.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.072+0000 7f862c83d700 1 -- 172.21.15.12:0/1169089394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f862806e2a0 msgr2=0x7f86281147e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:01.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.072+0000 7f862c83d700 1 --2- 172.21.15.12:0/1169089394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f862806e2a0 0x7f86281147e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f861c014040 tx=0x7f861c004770 comp rx=0 tx=0).stop 2024-09-13T23:43:01.079 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.072+0000 7f862c83d700 1 -- 172.21.15.12:0/1169089394 shutdown_connections 2024-09-13T23:43:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.072+0000 7f862c83d700 1 --2- 172.21.15.12:0/1169089394 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f86140354f0 0x7f86140379b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.072+0000 7f862c83d700 1 --2- 172.21.15.12:0/1169089394 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f862806e2a0 0x7f86281147e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.072+0000 7f862c83d700 1 -- 172.21.15.12:0/1169089394 >> 172.21.15.12:0/1169089394 conn(0x7f862809b0f0 msgr2=0x7f862806c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.072+0000 7f862c83d700 1 -- 172.21.15.12:0/1169089394 shutdown_connections 2024-09-13T23:43:01.080 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.072+0000 7f862c83d700 1 -- 172.21.15.12:0/1169089394 wait complete. 2024-09-13T23:43:01.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640260 for osd.3 2024-09-13T23:43:01.117 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.112+0000 7feb78663700 1 -- 172.21.15.12:0/19549361 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7feb7011e400 con 0x7feb70129490 2024-09-13T23:43:01.118 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.112+0000 7feb66ffd700 1 -- 172.21.15.12:0/19549361 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feb60011810 con 0x7feb70129490 2024-09-13T23:43:01.118 INFO:teuthology.orchestra.run.smithi012.stdout:42949672965 2024-09-13T23:43:01.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.116+0000 7feb78663700 1 -- 172.21.15.12:0/19549361 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7feb5c0354f0 msgr2=0x7feb5c0379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:01.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.116+0000 7feb78663700 1 --2- 172.21.15.12:0/19549361 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7feb5c0354f0 0x7feb5c0379b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb6800d370 tx=0x7feb68012000 comp rx=0 tx=0).stop 2024-09-13T23:43:01.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.116+0000 7feb78663700 1 -- 172.21.15.12:0/19549361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb70129490 msgr2=0x7feb701c19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:01.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.116+0000 7feb78663700 1 --2- 172.21.15.12:0/19549361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb70129490 0x7feb701c19f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7feb6000c480 tx=0x7feb6000c840 comp rx=0 tx=0).stop 2024-09-13T23:43:01.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.116+0000 7feb78663700 1 -- 172.21.15.12:0/19549361 shutdown_connections 2024-09-13T23:43:01.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.116+0000 7feb78663700 1 --2- 172.21.15.12:0/19549361 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7feb5c0354f0 0x7feb5c0379b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:01.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.116+0000 7feb78663700 1 --2- 172.21.15.12:0/19549361 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7feb70129490 0x7feb701c19f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:01.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.116+0000 7feb78663700 1 -- 172.21.15.12:0/19549361 >> 172.21.15.12:0/19549361 conn(0x7feb70099420 msgr2=0x7feb7006cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:01.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.116+0000 7feb78663700 1 -- 172.21.15.12:0/19549361 shutdown_connections 2024-09-13T23:43:01.122 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.116+0000 7feb78663700 1 -- 172.21.15.12:0/19549361 wait complete. 2024-09-13T23:43:01.134 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.128+0000 7f49db59e700 1 -- 172.21.15.12:0/1604200480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f49dc12a9d0 con 0x7f49dc12d0e0 2024-09-13T23:43:01.135 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.128+0000 7f49d2ffd700 1 -- 172.21.15.12:0/1604200480 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f49c4010380 con 0x7f49dc12d0e0 2024-09-13T23:43:01.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672967 got 42949672965 for osd.1 2024-09-13T23:43:01.136 INFO:teuthology.orchestra.run.smithi012.stdout:42949672965 2024-09-13T23:43:01.137 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.132+0000 7f49db59e700 1 -- 172.21.15.12:0/1604200480 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f49bc035540 msgr2=0x7f49bc037a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.132+0000 7f49db59e700 1 --2- 172.21.15.12:0/1604200480 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f49bc035540 0x7f49bc037a00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f49cc000c00 tx=0x7f49cc007a20 comp rx=0 tx=0).stop 2024-09-13T23:43:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.132+0000 7f49db59e700 1 -- 172.21.15.12:0/1604200480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49dc12d0e0 msgr2=0x7f49dc11ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.132+0000 7f49db59e700 1 --2- 172.21.15.12:0/1604200480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49dc12d0e0 0x7f49dc11ad30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f49c40090b0 tx=0x7f49c4004770 comp rx=0 tx=0).stop 2024-09-13T23:43:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.132+0000 7f49db59e700 1 -- 172.21.15.12:0/1604200480 shutdown_connections 2024-09-13T23:43:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.132+0000 7f49db59e700 1 --2- 172.21.15.12:0/1604200480 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f49bc035540 0x7f49bc037a00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.132+0000 7f49db59e700 1 --2- 172.21.15.12:0/1604200480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f49dc12d0e0 0x7f49dc11ad30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.132+0000 7f49db59e700 1 -- 172.21.15.12:0/1604200480 >> 172.21.15.12:0/1604200480 conn(0x7f49dc09cd60 msgr2=0x7f49dc1b50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.132+0000 7f49db59e700 1 -- 172.21.15.12:0/1604200480 shutdown_connections 2024-09-13T23:43:01.138 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.132+0000 7f49db59e700 1 -- 172.21.15.12:0/1604200480 wait complete. 2024-09-13T23:43:01.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672965 for osd.0 2024-09-13T23:43:01.157 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.152+0000 7fd2d159c700 1 -- 172.21.15.12:0/2469110386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd2cc06be70 con 0x7fd2cc06e480 2024-09-13T23:43:01.158 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.152+0000 7fd2c8ff9700 1 -- 172.21.15.12:0/2469110386 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd2b4017030 con 0x7fd2cc06e480 2024-09-13T23:43:01.158 INFO:teuthology.orchestra.run.smithi012.stdout:47244640261 2024-09-13T23:43:01.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.156+0000 7fd2d159c700 1 -- 172.21.15.12:0/2469110386 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fd2ac0354f0 msgr2=0x7fd2ac0379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:01.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.156+0000 7fd2d159c700 1 --2- 172.21.15.12:0/2469110386 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fd2ac0354f0 0x7fd2ac0379b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc070b10 tx=0x7fd2bc010000 comp rx=0 tx=0).stop 2024-09-13T23:43:01.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.156+0000 7fd2d159c700 1 -- 172.21.15.12:0/2469110386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2cc06e480 msgr2=0x7fd2cc11d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:01.161 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.156+0000 7fd2d159c700 1 --2- 172.21.15.12:0/2469110386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2cc06e480 0x7fd2cc11d3d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd2b400c680 tx=0x7fd2b400ca40 comp rx=0 tx=0).stop 2024-09-13T23:43:01.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.156+0000 7fd2d159c700 1 -- 172.21.15.12:0/2469110386 shutdown_connections 2024-09-13T23:43:01.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.156+0000 7fd2d159c700 1 --2- 172.21.15.12:0/2469110386 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fd2ac0354f0 0x7fd2ac0379b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:01.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.156+0000 7fd2d159c700 1 --2- 172.21.15.12:0/2469110386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd2cc06e480 0x7fd2cc11d3d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:01.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.156+0000 7fd2d159c700 1 -- 172.21.15.12:0/2469110386 >> 172.21.15.12:0/2469110386 conn(0x7fd2cc0991a0 msgr2=0x7fd2cc115c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:01.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.156+0000 7fd2d159c700 1 -- 172.21.15.12:0/2469110386 shutdown_connections 2024-09-13T23:43:01.162 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:01.156+0000 7fd2d159c700 1 -- 172.21.15.12:0/2469110386 wait complete. 2024-09-13T23:43:01.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640263 got 47244640261 for osd.2 2024-09-13T23:43:02.093 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T23:43:02.137 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T23:43:02.149 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T23:43:02.173 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T23:43:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.240+0000 7f3009fc4700 1 Processor -- start 2024-09-13T23:43:02.246 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.240+0000 7f3009fc4700 1 -- start start 2024-09-13T23:43:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.240+0000 7f3009fc4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f300406d5f0 0x7f300406d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.240+0000 7f3009fc4700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3004071200 con 0x7f300406df10 2024-09-13T23:43:02.247 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.240+0000 7f3009fc4700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300404ecd0 con 0x7f300406d5f0 2024-09-13T23:43:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.240+0000 7f3008fc2700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f300406d5f0 0x7f300406d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.240+0000 7f3003fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f300406df10 0x7f3004095060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48788/0 (socket says 172.21.15.12:48788) 2024-09-13T23:43:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.240+0000 7f3003fff700 1 -- 172.21.15.12:0/2299031504 learned_addr learned my addr 172.21.15.12:0/2299031504 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3008fc2700 1 -- 172.21.15.12:0/2299031504 >> v1:172.21.15.12:6789/0 conn(0x7f300406df10 legacy=0x7f3004095060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3008fc2700 1 -- 172.21.15.12:0/2299031504 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3004070910 con 0x7f300406d5f0 2024-09-13T23:43:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f30037fe700 1 -- 172.21.15.12:0/2299031504 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 608760621 0 0) 0x7f3004071200 con 0x7f300406df10 2024-09-13T23:43:02.248 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3008fc2700 1 --2- 172.21.15.12:0/2299031504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f300406d5f0 0x7f300406d9d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009d90 tx=0x7f2ff400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=859d47fa1b4e10dd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f30037fe700 1 -- 172.21.15.12:0/2299031504 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ff400e070 con 0x7f300406d5f0 2024-09-13T23:43:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f30037fe700 1 -- 172.21.15.12:0/2299031504 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff400bc80 con 0x7f300406d5f0 2024-09-13T23:43:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f30037fe700 1 -- 172.21.15.12:0/2299031504 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ff4003a40 con 0x7f300406d5f0 2024-09-13T23:43:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3009fc4700 1 -- 172.21.15.12:0/2299031504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f300406d5f0 msgr2=0x7f300406d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3009fc4700 1 --2- 172.21.15.12:0/2299031504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f300406d5f0 0x7f300406d9d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009d90 tx=0x7f2ff400b330 comp rx=0 tx=0).stop 2024-09-13T23:43:02.249 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3009fc4700 1 -- 172.21.15.12:0/2299031504 shutdown_connections 2024-09-13T23:43:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3009fc4700 1 --2- 172.21.15.12:0/2299031504 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f300406d5f0 0x7f300406d9d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3009fc4700 1 -- 172.21.15.12:0/2299031504 >> 172.21.15.12:0/2299031504 conn(0x7f30040988a0 msgr2=0x7f3004098cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3009fc4700 1 -- 172.21.15.12:0/2299031504 shutdown_connections 2024-09-13T23:43:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3009fc4700 1 -- 172.21.15.12:0/2299031504 wait complete. 2024-09-13T23:43:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3009fc4700 1 Processor -- start 2024-09-13T23:43:02.250 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3009fc4700 1 -- start start 2024-09-13T23:43:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3009fc4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f300406d5f0 0x7f300406e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.244+0000 7f3009fc4700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300406fc50 con 0x7f300406d5f0 2024-09-13T23:43:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f3008fc2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f300406d5f0 0x7f300406e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f3008fc2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f300406d5f0 0x7f300406e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39544/0 (socket says 172.21.15.12:39544) 2024-09-13T23:43:02.251 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f3008fc2700 1 -- 172.21.15.12:0/1584794214 learned_addr learned my addr 172.21.15.12:0/1584794214 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:02.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f3008fc2700 1 -- 172.21.15.12:0/1584794214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff4009a70 con 0x7f300406d5f0 2024-09-13T23:43:02.252 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f3008fc2700 1 --2- 172.21.15.12:0/1584794214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f300406d5f0 0x7f300406e700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009470 tx=0x7f2ff4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f30017fa700 1 -- 172.21.15.12:0/1584794214 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ff4004710 con 0x7f300406d5f0 2024-09-13T23:43:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f3009fc4700 1 -- 172.21.15.12:0/1584794214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300406fe30 con 0x7f300406d5f0 2024-09-13T23:43:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f30017fa700 1 -- 172.21.15.12:0/1584794214 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff4004890 con 0x7f300406d5f0 2024-09-13T23:43:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f30017fa700 1 -- 172.21.15.12:0/1584794214 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ff40125a0 con 0x7f300406d5f0 2024-09-13T23:43:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f3009fc4700 1 -- 172.21.15.12:0/1584794214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300412a530 con 0x7f300406d5f0 2024-09-13T23:43:02.253 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f30017fa700 1 -- 172.21.15.12:0/1584794214 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f2ff4012700 con 0x7f300406d5f0 2024-09-13T23:43:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f30017fa700 1 --2- 172.21.15.12:0/1584794214 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f2ff00354a0 0x7f2ff0037960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f30017fa700 1 -- 172.21.15.12:0/1584794214 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f2ff4049c40 con 0x7f300406d5f0 2024-09-13T23:43:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f3009fc4700 1 -- 172.21.15.12:0/1584794214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3004071da0 con 0x7f300406d5f0 2024-09-13T23:43:02.254 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f3003fff700 1 --2- 172.21.15.12:0/1584794214 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f2ff00354a0 0x7f2ff0037960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.255 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.248+0000 7f3003fff700 1 --2- 172.21.15.12:0/1584794214 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f2ff00354a0 0x7f2ff0037960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8000c00 tx=0x7f2ff8007a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.261 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.256+0000 7f30017fa700 1 -- 172.21.15.12:0/1584794214 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2ff4011390 con 0x7f300406d5f0 2024-09-13T23:43:02.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d3dc98700 1 Processor -- start 2024-09-13T23:43:02.264 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d3dc98700 1 -- start start 2024-09-13T23:43:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d3dc98700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d3812deb0 0x7f5d381302a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d3dc98700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d381275c0 con 0x7f5d381279f0 2024-09-13T23:43:02.265 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d3dc98700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d381284e0 con 0x7f5d3812deb0 2024-09-13T23:43:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d37fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d3812deb0 0x7f5d381302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d3cc96700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5d381279f0 0x7f5d38127dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48800/0 (socket says 172.21.15.12:48800) 2024-09-13T23:43:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d3cc96700 1 -- 172.21.15.12:0/352333485 learned_addr learned my addr 172.21.15.12:0/352333485 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d377fe700 1 -- 172.21.15.12:0/352333485 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721835800 0 0) 0x7f5d381275c0 con 0x7f5d381279f0 2024-09-13T23:43:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d37fff700 1 -- 172.21.15.12:0/352333485 >> v1:172.21.15.12:6789/0 conn(0x7f5d381279f0 legacy=0x7f5d38127dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.266 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d37fff700 1 -- 172.21.15.12:0/352333485 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d38111880 con 0x7f5d3812deb0 2024-09-13T23:43:02.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d37fff700 1 --2- 172.21.15.12:0/352333485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d3812deb0 0x7f5d381302a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5d240097d0 tx=0x7f5d24009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea09d830b093a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d377fe700 1 -- 172.21.15.12:0/352333485 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d240041c0 con 0x7f5d3812deb0 2024-09-13T23:43:02.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d377fe700 1 -- 172.21.15.12:0/352333485 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d24004320 con 0x7f5d3812deb0 2024-09-13T23:43:02.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d377fe700 1 -- 172.21.15.12:0/352333485 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d240036c0 con 0x7f5d3812deb0 2024-09-13T23:43:02.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d3dc98700 1 -- 172.21.15.12:0/352333485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d3812deb0 msgr2=0x7f5d381302a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.267 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.260+0000 7f5d3dc98700 1 --2- 172.21.15.12:0/352333485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d3812deb0 0x7f5d381302a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5d240097d0 tx=0x7f5d24009ae0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3dc98700 1 -- 172.21.15.12:0/352333485 shutdown_connections 2024-09-13T23:43:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3dc98700 1 --2- 172.21.15.12:0/352333485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d3812deb0 0x7f5d381302a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3dc98700 1 -- 172.21.15.12:0/352333485 >> 172.21.15.12:0/352333485 conn(0x7f5d380991a0 msgr2=0x7f5d380995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3dc98700 1 -- 172.21.15.12:0/352333485 shutdown_connections 2024-09-13T23:43:02.268 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3dc98700 1 -- 172.21.15.12:0/352333485 wait complete. 2024-09-13T23:43:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3dc98700 1 Processor -- start 2024-09-13T23:43:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3dc98700 1 -- start start 2024-09-13T23:43:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3dc98700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d381279f0 0x7f5d3806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3dc98700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d38071890 con 0x7f5d381279f0 2024-09-13T23:43:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3cc96700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d381279f0 0x7f5d3806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3cc96700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d381279f0 0x7f5d3806ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39566/0 (socket says 172.21.15.12:39566) 2024-09-13T23:43:02.269 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3cc96700 1 -- 172.21.15.12:0/1879154270 learned_addr learned my addr 172.21.15.12:0/1879154270 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3cc96700 1 -- 172.21.15.12:0/1879154270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d240094b0 con 0x7f5d381279f0 2024-09-13T23:43:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3cc96700 1 --2- 172.21.15.12:0/1879154270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d381279f0 0x7f5d3806ffe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00c4d0 tx=0x7f5d2c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d357fa700 1 -- 172.21.15.12:0/1879154270 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d2c004bd0 con 0x7f5d381279f0 2024-09-13T23:43:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d357fa700 1 -- 172.21.15.12:0/1879154270 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c004d50 con 0x7f5d381279f0 2024-09-13T23:43:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3dc98700 1 -- 172.21.15.12:0/1879154270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3809b030 con 0x7f5d381279f0 2024-09-13T23:43:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d357fa700 1 -- 172.21.15.12:0/1879154270 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d2c0070d0 con 0x7f5d381279f0 2024-09-13T23:43:02.270 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.264+0000 7f5d3dc98700 1 -- 172.21.15.12:0/1879154270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d38117190 con 0x7f5d381279f0 2024-09-13T23:43:02.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.268+0000 7f5d357fa700 1 -- 172.21.15.12:0/1879154270 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f5d2c007280 con 0x7f5d381279f0 2024-09-13T23:43:02.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.268+0000 7f5d3dc98700 1 -- 172.21.15.12:0/1879154270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d381181a0 con 0x7f5d381279f0 2024-09-13T23:43:02.271 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.268+0000 7f5d357fa700 1 --2- 172.21.15.12:0/1879154270 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f5d1c0354f0 0x7f5d1c0379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.268+0000 7f5d357fa700 1 -- 172.21.15.12:0/1879154270 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f5d2c0473f0 con 0x7f5d381279f0 2024-09-13T23:43:02.272 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.268+0000 7f5d37fff700 1 --2- 172.21.15.12:0/1879154270 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f5d1c0354f0 0x7f5d1c0379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.273 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.268+0000 7f5d37fff700 1 --2- 172.21.15.12:0/1879154270 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f5d1c0354f0 0x7f5d1c0379b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5d24014040 tx=0x7f5d24010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.272+0000 7f8fa63d2700 1 Processor -- start 2024-09-13T23:43:02.278 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.272+0000 7f8fa63d2700 1 -- start start 2024-09-13T23:43:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.272+0000 7f5d357fa700 1 -- 172.21.15.12:0/1879154270 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5d381181a0 con 0x7f5d381279f0 2024-09-13T23:43:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8fa012deb0 0x7f8fa01302a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.279 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa01275c0 con 0x7f8fa01279f0 2024-09-13T23:43:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa01284e0 con 0x7f8fa012deb0 2024-09-13T23:43:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa4bcf700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f8fa012deb0 0x7f8fa01302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa53d0700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f8fa01279f0 0x7f8fa0127dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48814/0 (socket says 172.21.15.12:48814) 2024-09-13T23:43:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa53d0700 1 -- 172.21.15.12:0/3206019194 learned_addr learned my addr 172.21.15.12:0/3206019194 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:02.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa4bcf700 1 -- 172.21.15.12:0/3206019194 >> v1:172.21.15.12:6789/0 conn(0x7f8fa01279f0 legacy=0x7f8fa0127dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa4bcf700 1 -- 172.21.15.12:0/3206019194 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa0111880 con 0x7f8fa012deb0 2024-09-13T23:43:02.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa4bcf700 1 --2- 172.21.15.12:0/3206019194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fa012deb0 0x7f8fa01302a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8f900097d0 tx=0x7f8f90009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2de1369b8234d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8f97fff700 1 -- 172.21.15.12:0/3206019194 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f900041c0 con 0x7f8fa012deb0 2024-09-13T23:43:02.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8f97fff700 1 -- 172.21.15.12:0/3206019194 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f90004320 con 0x7f8fa012deb0 2024-09-13T23:43:02.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8f97fff700 1 -- 172.21.15.12:0/3206019194 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f900036c0 con 0x7f8fa012deb0 2024-09-13T23:43:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3206019194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fa012deb0 msgr2=0x7f8fa01302a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 --2- 172.21.15.12:0/3206019194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fa012deb0 0x7f8fa01302a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8f900097d0 tx=0x7f8f90009ae0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3206019194 shutdown_connections 2024-09-13T23:43:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 --2- 172.21.15.12:0/3206019194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fa012deb0 0x7f8fa01302a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3206019194 >> 172.21.15.12:0/3206019194 conn(0x7f8fa00991a0 msgr2=0x7f8fa00995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3206019194 shutdown_connections 2024-09-13T23:43:02.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3206019194 wait complete. 2024-09-13T23:43:02.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 Processor -- start 2024-09-13T23:43:02.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 -- start start 2024-09-13T23:43:02.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fa01279f0 0x7f8fa01bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.276+0000 7f8fa63d2700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa0121bf0 con 0x7f8fa01279f0 2024-09-13T23:43:02.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8fa53d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fa01279f0 0x7f8fa01bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8fa53d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fa01279f0 0x7f8fa01bf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39596/0 (socket says 172.21.15.12:39596) 2024-09-13T23:43:02.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8fa53d0700 1 -- 172.21.15.12:0/3899038986 learned_addr learned my addr 172.21.15.12:0/3899038986 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:02.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8fa53d0700 1 -- 172.21.15.12:0/3899038986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900094b0 con 0x7f8fa01279f0 2024-09-13T23:43:02.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8fa53d0700 1 --2- 172.21.15.12:0/3899038986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fa01279f0 0x7f8fa01bf4f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800ca90 tx=0x7f8f9800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8f95ffb700 1 -- 172.21.15.12:0/3899038986 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f980073b0 con 0x7f8fa01279f0 2024-09-13T23:43:02.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8f95ffb700 1 -- 172.21.15.12:0/3899038986 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f98007510 con 0x7f8fa01279f0 2024-09-13T23:43:02.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3899038986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa009b030 con 0x7f8fa01279f0 2024-09-13T23:43:02.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8f95ffb700 1 -- 172.21.15.12:0/3899038986 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f980077b0 con 0x7f8fa01279f0 2024-09-13T23:43:02.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3899038986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa0112c70 con 0x7f8fa01279f0 2024-09-13T23:43:02.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8f95ffb700 1 -- 172.21.15.12:0/3899038986 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f8f98007910 con 0x7f8fa01279f0 2024-09-13T23:43:02.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8f95ffb700 1 --2- 172.21.15.12:0/3899038986 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f8f840354f0 0x7f8f840379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8f95ffb700 1 -- 172.21.15.12:0/3899038986 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f8f98047420 con 0x7f8fa01279f0 2024-09-13T23:43:02.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8fa4bcf700 1 --2- 172.21.15.12:0/3899038986 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f8f840354f0 0x7f8f840379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.280+0000 7f8fa4bcf700 1 --2- 172.21.15.12:0/3899038986 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f8f840354f0 0x7f8f840379b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8f90014040 tx=0x7f8f90010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.284+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3899038986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa012add0 con 0x7f8fa01279f0 2024-09-13T23:43:02.291 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.288+0000 7f3e85192700 1 Processor -- start 2024-09-13T23:43:02.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.288+0000 7f3e85192700 1 -- start start 2024-09-13T23:43:02.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.288+0000 7f3e85192700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3e80070700 0x7f3e80132300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.292 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.288+0000 7f3e85192700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8006f8c0 con 0x7f3e8006e480 2024-09-13T23:43:02.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.288+0000 7f3e85192700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e8006fc90 con 0x7f3e80070700 2024-09-13T23:43:02.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.288+0000 7f3e7f7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3e80070700 0x7f3e80132300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.288+0000 7f3e7f7fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3e80070700 0x7f3e80132300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39604/0 (socket says 172.21.15.12:39604) 2024-09-13T23:43:02.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.288+0000 7f3e7f7fe700 1 -- 172.21.15.12:0/3223009953 learned_addr learned my addr 172.21.15.12:0/3223009953 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:02.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.288+0000 7f3e7f7fe700 1 -- 172.21.15.12:0/3223009953 >> v1:172.21.15.12:6789/0 conn(0x7f3e8006e480 legacy=0x7f3e8006e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.288+0000 7f3e7f7fe700 1 -- 172.21.15.12:0/3223009953 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8009af40 con 0x7f3e80070700 2024-09-13T23:43:02.293 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.288+0000 7f3e7f7fe700 1 --2- 172.21.15.12:0/3223009953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e80070700 0x7f3e80132300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3e78009e50 tx=0x7f3e7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=213ca3fba371be97 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e7effd700 1 -- 172.21.15.12:0/3223009953 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e7800e070 con 0x7f3e80070700 2024-09-13T23:43:02.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e7effd700 1 -- 172.21.15.12:0/3223009953 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e78003a40 con 0x7f3e80070700 2024-09-13T23:43:02.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e7effd700 1 -- 172.21.15.12:0/3223009953 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e78003d00 con 0x7f3e80070700 2024-09-13T23:43:02.296 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f8f95ffb700 1 -- 172.21.15.12:0/3899038986 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8f98018070 con 0x7f8fa01279f0 2024-09-13T23:43:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e85192700 1 -- 172.21.15.12:0/3223009953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e80070700 msgr2=0x7f3e80132300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e85192700 1 --2- 172.21.15.12:0/3223009953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e80070700 0x7f3e80132300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3e78009e50 tx=0x7f3e7800b330 comp rx=0 tx=0).stop 2024-09-13T23:43:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e85192700 1 -- 172.21.15.12:0/3223009953 shutdown_connections 2024-09-13T23:43:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e85192700 1 --2- 172.21.15.12:0/3223009953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e80070700 0x7f3e80132300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e85192700 1 -- 172.21.15.12:0/3223009953 >> 172.21.15.12:0/3223009953 conn(0x7f3e800991a0 msgr2=0x7f3e800995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e85192700 1 -- 172.21.15.12:0/3223009953 shutdown_connections 2024-09-13T23:43:02.297 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e85192700 1 -- 172.21.15.12:0/3223009953 wait complete. 2024-09-13T23:43:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e85192700 1 Processor -- start 2024-09-13T23:43:02.298 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e85192700 1 -- start start 2024-09-13T23:43:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e85192700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e8006e480 0x7f3e8011d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e85192700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e801205f0 con 0x7f3e8006e480 2024-09-13T23:43:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.292+0000 7f3e7ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e8006e480 0x7f3e8011d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e7ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e8006e480 0x7f3e8011d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39612/0 (socket says 172.21.15.12:39612) 2024-09-13T23:43:02.299 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e7ffff700 1 -- 172.21.15.12:0/3093948349 learned_addr learned my addr 172.21.15.12:0/3093948349 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e7ffff700 1 -- 172.21.15.12:0/3093948349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e78009b30 con 0x7f3e8006e480 2024-09-13T23:43:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e7ffff700 1 --2- 172.21.15.12:0/3093948349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e8006e480 0x7f3e8011d3d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400c480 tx=0x7f3e7400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e7cff9700 1 -- 172.21.15.12:0/3093948349 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e74005150 con 0x7f3e8006e480 2024-09-13T23:43:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e85192700 1 -- 172.21.15.12:0/3093948349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8011ebb0 con 0x7f3e8006e480 2024-09-13T23:43:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e85192700 1 -- 172.21.15.12:0/3093948349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8011fd50 con 0x7f3e8006e480 2024-09-13T23:43:02.300 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e7cff9700 1 -- 172.21.15.12:0/3093948349 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e740052b0 con 0x7f3e8006e480 2024-09-13T23:43:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e7cff9700 1 -- 172.21.15.12:0/3093948349 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e74005590 con 0x7f3e8006e480 2024-09-13T23:43:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e85192700 1 -- 172.21.15.12:0/3093948349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6c005100 con 0x7f3e8006e480 2024-09-13T23:43:02.301 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e7cff9700 1 -- 172.21.15.12:0/3093948349 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f3e74006880 con 0x7f3e8006e480 2024-09-13T23:43:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e7cff9700 1 --2- 172.21.15.12:0/3093948349 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f3e680354f0 0x7f3e680379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e7cff9700 1 -- 172.21.15.12:0/3093948349 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f3e74046ad0 con 0x7f3e8006e480 2024-09-13T23:43:02.302 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.296+0000 7f3e7f7fe700 1 --2- 172.21.15.12:0/3093948349 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f3e680354f0 0x7f3e680379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.303 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.300+0000 7f3e7f7fe700 1 --2- 172.21.15.12:0/3093948349 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f3e680354f0 0x7f3e680379b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3e80070b10 tx=0x7f3e78009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.310 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.304+0000 7f3e7cff9700 1 -- 172.21.15.12:0/3093948349 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3e740102e0 con 0x7f3e8006e480 2024-09-13T23:43:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.440+0000 7f3009fc4700 1 -- 172.21.15.12:0/1584794214 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f300406d9d0 con 0x7f300406d5f0 2024-09-13T23:43:02.445 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.440+0000 7f30017fa700 1 -- 172.21.15.12:0/1584794214 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2ff4032070 con 0x7f300406d5f0 2024-09-13T23:43:02.446 INFO:teuthology.orchestra.run.smithi012.stdout:47244640262 2024-09-13T23:43:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.444+0000 7f3009fc4700 1 -- 172.21.15.12:0/1584794214 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f2ff00354a0 msgr2=0x7f2ff0037960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.444+0000 7f3009fc4700 1 --2- 172.21.15.12:0/1584794214 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f2ff00354a0 0x7f2ff0037960 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8000c00 tx=0x7f2ff8007a20 comp rx=0 tx=0).stop 2024-09-13T23:43:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.444+0000 7f3009fc4700 1 -- 172.21.15.12:0/1584794214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f300406d5f0 msgr2=0x7f300406e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.448 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.444+0000 7f3009fc4700 1 --2- 172.21.15.12:0/1584794214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f300406d5f0 0x7f300406e700 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4009470 tx=0x7f2ff4004af0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.444+0000 7f3009fc4700 1 -- 172.21.15.12:0/1584794214 shutdown_connections 2024-09-13T23:43:02.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.444+0000 7f3009fc4700 1 --2- 172.21.15.12:0/1584794214 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f2ff00354a0 0x7f2ff0037960 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.444+0000 7f3009fc4700 1 --2- 172.21.15.12:0/1584794214 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f300406d5f0 0x7f300406e700 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.444+0000 7f3009fc4700 1 -- 172.21.15.12:0/1584794214 >> 172.21.15.12:0/1584794214 conn(0x7f30040988a0 msgr2=0x7f300412b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:02.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.444+0000 7f3009fc4700 1 -- 172.21.15.12:0/1584794214 shutdown_connections 2024-09-13T23:43:02.449 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.444+0000 7f3009fc4700 1 -- 172.21.15.12:0/1584794214 wait complete. 2024-09-13T23:43:02.452 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.448+0000 7f5d3dc98700 1 -- 172.21.15.12:0/1879154270 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5d38118750 con 0x7f5d381279f0 2024-09-13T23:43:02.453 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.448+0000 7f5d357fa700 1 -- 172.21.15.12:0/1879154270 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5d38118750 con 0x7f5d381279f0 2024-09-13T23:43:02.453 INFO:teuthology.orchestra.run.smithi012.stdout:42949672966 2024-09-13T23:43:02.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.452+0000 7f5d3dc98700 1 -- 172.21.15.12:0/1879154270 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f5d1c0354f0 msgr2=0x7f5d1c0379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.452+0000 7f5d3dc98700 1 --2- 172.21.15.12:0/1879154270 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f5d1c0354f0 0x7f5d1c0379b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5d24014040 tx=0x7f5d24010000 comp rx=0 tx=0).stop 2024-09-13T23:43:02.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.452+0000 7f5d3dc98700 1 -- 172.21.15.12:0/1879154270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d381279f0 msgr2=0x7f5d3806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.452+0000 7f5d3dc98700 1 --2- 172.21.15.12:0/1879154270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d381279f0 0x7f5d3806ffe0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00c4d0 tx=0x7f5d2c00c890 comp rx=0 tx=0).stop 2024-09-13T23:43:02.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.452+0000 7f5d3dc98700 1 -- 172.21.15.12:0/1879154270 shutdown_connections 2024-09-13T23:43:02.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.452+0000 7f5d3dc98700 1 --2- 172.21.15.12:0/1879154270 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f5d1c0354f0 0x7f5d1c0379b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.452+0000 7f5d3dc98700 1 --2- 172.21.15.12:0/1879154270 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d381279f0 0x7f5d3806ffe0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.452+0000 7f5d3dc98700 1 -- 172.21.15.12:0/1879154270 >> 172.21.15.12:0/1879154270 conn(0x7f5d380991a0 msgr2=0x7f5d3806c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:02.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.452+0000 7f5d3dc98700 1 -- 172.21.15.12:0/1879154270 shutdown_connections 2024-09-13T23:43:02.456 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.452+0000 7f5d3dc98700 1 -- 172.21.15.12:0/1879154270 wait complete. 2024-09-13T23:43:02.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640262 got 47244640262 for osd.3 2024-09-13T23:43:02.460 DEBUG:teuthology.parallel:result is None 2024-09-13T23:43:02.466 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.0 2024-09-13T23:43:02.467 DEBUG:teuthology.parallel:result is None 2024-09-13T23:43:02.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.492+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3899038986 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8fa01125a0 con 0x7f8fa01279f0 2024-09-13T23:43:02.498 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.492+0000 7f8f95ffb700 1 -- 172.21.15.12:0/3899038986 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8f9800a920 con 0x7f8fa01279f0 2024-09-13T23:43:02.498 INFO:teuthology.orchestra.run.smithi012.stdout:42949672967 2024-09-13T23:43:02.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.496+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3899038986 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f8f840354f0 msgr2=0x7f8f840379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.500 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.496+0000 7f8fa63d2700 1 --2- 172.21.15.12:0/3899038986 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f8f840354f0 0x7f8f840379b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8f90014040 tx=0x7f8f90010000 comp rx=0 tx=0).stop 2024-09-13T23:43:02.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.496+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3899038986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fa01279f0 msgr2=0x7f8fa01bf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.496+0000 7f8fa63d2700 1 --2- 172.21.15.12:0/3899038986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fa01279f0 0x7f8fa01bf4f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f9800ca90 tx=0x7f8f9800cb70 comp rx=0 tx=0).stop 2024-09-13T23:43:02.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.496+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3899038986 shutdown_connections 2024-09-13T23:43:02.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.496+0000 7f8fa63d2700 1 --2- 172.21.15.12:0/3899038986 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f8f840354f0 0x7f8f840379b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.496+0000 7f8fa63d2700 1 --2- 172.21.15.12:0/3899038986 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f8fa01279f0 0x7f8fa01bf4f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.496+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3899038986 >> 172.21.15.12:0/3899038986 conn(0x7f8fa00991a0 msgr2=0x7f8fa01b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:02.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.496+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3899038986 shutdown_connections 2024-09-13T23:43:02.501 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.496+0000 7f8fa63d2700 1 -- 172.21.15.12:0/3899038986 wait complete. 2024-09-13T23:43:02.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672967 got 42949672967 for osd.1 2024-09-13T23:43:02.512 DEBUG:teuthology.parallel:result is None 2024-09-13T23:43:02.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.512+0000 7f3e85192700 1 -- 172.21.15.12:0/3093948349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3e6c0053b0 con 0x7f3e8006e480 2024-09-13T23:43:02.517 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.512+0000 7f3e7cff9700 1 -- 172.21.15.12:0/3093948349 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e740102e0 con 0x7f3e8006e480 2024-09-13T23:43:02.517 INFO:teuthology.orchestra.run.smithi012.stdout:47244640263 2024-09-13T23:43:02.519 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.516+0000 7f3e85192700 1 -- 172.21.15.12:0/3093948349 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f3e680354f0 msgr2=0x7f3e680379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.516+0000 7f3e85192700 1 --2- 172.21.15.12:0/3093948349 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f3e680354f0 0x7f3e680379b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3e80070b10 tx=0x7f3e78009000 comp rx=0 tx=0).stop 2024-09-13T23:43:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.516+0000 7f3e85192700 1 -- 172.21.15.12:0/3093948349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e8006e480 msgr2=0x7f3e8011d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.516+0000 7f3e85192700 1 --2- 172.21.15.12:0/3093948349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e8006e480 0x7f3e8011d3d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3e7400c480 tx=0x7f3e7400c840 comp rx=0 tx=0).stop 2024-09-13T23:43:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.516+0000 7f3e85192700 1 -- 172.21.15.12:0/3093948349 shutdown_connections 2024-09-13T23:43:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.516+0000 7f3e85192700 1 --2- 172.21.15.12:0/3093948349 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f3e680354f0 0x7f3e680379b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.516+0000 7f3e85192700 1 --2- 172.21.15.12:0/3093948349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3e8006e480 0x7f3e8011d3d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.516+0000 7f3e85192700 1 -- 172.21.15.12:0/3093948349 >> 172.21.15.12:0/3093948349 conn(0x7f3e800991a0 msgr2=0x7f3e80115c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.516+0000 7f3e85192700 1 -- 172.21.15.12:0/3093948349 shutdown_connections 2024-09-13T23:43:02.520 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.516+0000 7f3e85192700 1 -- 172.21.15.12:0/3093948349 wait complete. 2024-09-13T23:43:02.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640263 got 47244640263 for osd.2 2024-09-13T23:43:02.532 DEBUG:teuthology.parallel:result is None 2024-09-13T23:43:02.532 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T23:43:02.532 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T23:43:02.644 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f272a85d700 1 Processor -- start 2024-09-13T23:43:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f272a85d700 1 -- start start 2024-09-13T23:43:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f272a85d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f272412ad60 0x7f272412fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f272a85d700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f272406c230 con 0x7f272406f920 2024-09-13T23:43:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f272a85d700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2724111d80 con 0x7f272412ad60 2024-09-13T23:43:02.645 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f272985b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f272406f920 0x7f272412b210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48832/0 (socket says 172.21.15.12:48832) 2024-09-13T23:43:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f272905a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f272412ad60 0x7f272412fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f272985b700 1 -- 172.21.15.12:0/842033776 learned_addr learned my addr 172.21.15.12:0/842033776 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f272905a700 1 -- 172.21.15.12:0/842033776 >> v1:172.21.15.12:6789/0 conn(0x7f272406f920 legacy=0x7f272412b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f2728859700 1 -- 172.21.15.12:0/842033776 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121257135 0 0) 0x7f272406c230 con 0x7f272406f920 2024-09-13T23:43:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f272905a700 1 -- 172.21.15.12:0/842033776 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272412b920 con 0x7f272412ad60 2024-09-13T23:43:02.646 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f272905a700 1 --2- 172.21.15.12:0/842033776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f272412ad60 0x7f272412fe70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2720009d90 tx=0x7f272000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce43a7583e56287 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f2728859700 1 -- 172.21.15.12:0/842033776 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f272000e070 con 0x7f272412ad60 2024-09-13T23:43:02.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f2728859700 1 -- 172.21.15.12:0/842033776 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272000bc80 con 0x7f272412ad60 2024-09-13T23:43:02.647 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.640+0000 7f2728859700 1 -- 172.21.15.12:0/842033776 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2720003a40 con 0x7f272412ad60 2024-09-13T23:43:02.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.644+0000 7f272a85d700 1 -- 172.21.15.12:0/842033776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f272412ad60 msgr2=0x7f272412fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.644+0000 7f272a85d700 1 --2- 172.21.15.12:0/842033776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f272412ad60 0x7f272412fe70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2720009d90 tx=0x7f272000b330 comp rx=0 tx=0).stop 2024-09-13T23:43:02.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.644+0000 7f272a85d700 1 -- 172.21.15.12:0/842033776 shutdown_connections 2024-09-13T23:43:02.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.644+0000 7f272a85d700 1 --2- 172.21.15.12:0/842033776 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f272412ad60 0x7f272412fe70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.644+0000 7f272a85d700 1 -- 172.21.15.12:0/842033776 >> 172.21.15.12:0/842033776 conn(0x7f27240991a0 msgr2=0x7f27240995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:02.648 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.644+0000 7f272a85d700 1 -- 172.21.15.12:0/842033776 shutdown_connections 2024-09-13T23:43:02.649 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.644+0000 7f272a85d700 1 -- 172.21.15.12:0/842033776 wait complete. 2024-09-13T23:43:02.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.644+0000 7f272a85d700 1 Processor -- start 2024-09-13T23:43:02.650 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.644+0000 7f272a85d700 1 -- start start 2024-09-13T23:43:02.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.644+0000 7f272a85d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f272406f920 0x7f2724120580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.648+0000 7f272a85d700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272412f880 con 0x7f272406f920 2024-09-13T23:43:02.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.648+0000 7f272985b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f272406f920 0x7f2724120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.651 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.648+0000 7f272985b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f272406f920 0x7f2724120580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39626/0 (socket says 172.21.15.12:39626) 2024-09-13T23:43:02.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.648+0000 7f272985b700 1 -- 172.21.15.12:0/648372340 learned_addr learned my addr 172.21.15.12:0/648372340 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:02.652 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.648+0000 7f272985b700 1 -- 172.21.15.12:0/648372340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2720009a70 con 0x7f272406f920 2024-09-13T23:43:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.648+0000 7f272985b700 1 --2- 172.21.15.12:0/648372340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f272406f920 0x7f2724120580 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f271400c4d0 tx=0x7f271400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.648+0000 7f271a7fc700 1 -- 172.21.15.12:0/648372340 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2714004bd0 con 0x7f272406f920 2024-09-13T23:43:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.648+0000 7f271a7fc700 1 -- 172.21.15.12:0/648372340 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2714004d50 con 0x7f272406f920 2024-09-13T23:43:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.648+0000 7f272a85d700 1 -- 172.21.15.12:0/648372340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2724112660 con 0x7f272406f920 2024-09-13T23:43:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.648+0000 7f271a7fc700 1 -- 172.21.15.12:0/648372340 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27140070d0 con 0x7f272406f920 2024-09-13T23:43:02.653 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.648+0000 7f272a85d700 1 -- 172.21.15.12:0/648372340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2724113470 con 0x7f272406f920 2024-09-13T23:43:02.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.648+0000 7f271a7fc700 1 -- 172.21.15.12:0/648372340 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f27140072d0 con 0x7f272406f920 2024-09-13T23:43:02.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.652+0000 7f272a85d700 1 -- 172.21.15.12:0/648372340 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2724113720 con 0x7f272406f920 2024-09-13T23:43:02.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.652+0000 7f271a7fc700 1 --2- 172.21.15.12:0/648372340 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f27080354a0 0x7f2708037960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.655 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.652+0000 7f271a7fc700 1 -- 172.21.15.12:0/648372340 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f2714047430 con 0x7f272406f920 2024-09-13T23:43:02.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.652+0000 7f272905a700 1 --2- 172.21.15.12:0/648372340 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f27080354a0 0x7f2708037960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.656 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.652+0000 7f272905a700 1 --2- 172.21.15.12:0/648372340 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f27080354a0 0x7f2708037960 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2724128630 tx=0x7f2720011000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.662 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.656+0000 7f271a7fc700 1 -- 172.21.15.12:0/648372340 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f27140102c0 con 0x7f272406f920 2024-09-13T23:43:02.830 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.824+0000 7f272a85d700 1 -- 172.21.15.12:0/648372340 --> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2724113270 con 0x7f27080354a0 2024-09-13T23:43:02.833 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.828+0000 7f271a7fc700 1 -- 172.21.15.12:0/648372340 <== mgr.4099 v2:172.21.15.167:6824/1427656188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+35026 (secure 0 0 0) 0x7f2724113270 con 0x7f27080354a0 2024-09-13T23:43:02.834 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:43:02.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.832+0000 7f272a85d700 1 -- 172.21.15.12:0/648372340 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f27080354a0 msgr2=0x7f2708037960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.836 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.832+0000 7f272a85d700 1 --2- 172.21.15.12:0/648372340 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f27080354a0 0x7f2708037960 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2724128630 tx=0x7f2720011000 comp rx=0 tx=0).stop 2024-09-13T23:43:02.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.832+0000 7f272a85d700 1 -- 172.21.15.12:0/648372340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f272406f920 msgr2=0x7f2724120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.832+0000 7f272a85d700 1 --2- 172.21.15.12:0/648372340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f272406f920 0x7f2724120580 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f271400c4d0 tx=0x7f271400c890 comp rx=0 tx=0).stop 2024-09-13T23:43:02.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.832+0000 7f272a85d700 1 -- 172.21.15.12:0/648372340 shutdown_connections 2024-09-13T23:43:02.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.832+0000 7f272a85d700 1 --2- 172.21.15.12:0/648372340 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f27080354a0 0x7f2708037960 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.832+0000 7f272a85d700 1 --2- 172.21.15.12:0/648372340 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f272406f920 0x7f2724120580 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.832+0000 7f272a85d700 1 -- 172.21.15.12:0/648372340 >> 172.21.15.12:0/648372340 conn(0x7f27240991a0 msgr2=0x7f272406e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:02.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.832+0000 7f272a85d700 1 -- 172.21.15.12:0/648372340 shutdown_connections 2024-09-13T23:43:02.837 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.832+0000 7f272a85d700 1 -- 172.21.15.12:0/648372340 wait complete. 2024-09-13T23:43:02.837 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-13T23:43:02.848 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-13T23:43:01.460082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":118044,"kb_used_data":1772,"kb_used_omap":6,"kb_used_meta":116217,"kb_avail":377369316,"statfs":{"total":386547056640,"available":386426179584,"internally_reserved":0,"allocated":1814528,"data_stored":1378695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6374,"internal_metadata":119007002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.939755"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.554661+0000","last_change":"2024-09-13T23:42:59.554946+0000","last_active":"2024-09-13T23:42:59.554661+0000","last_peered":"2024-09-13T23:42:59.554661+0000","last_clean":"2024-09-13T23:42:59.554661+0000","last_became_active":"2024-09-13T23:42:57.544266+0000","last_became_peered":"2024-09-13T23:42:57.544266+0000","last_unstale":"2024-09-13T23:42:59.554661+0000","last_undegraded":"2024-09-13T23:42:59.554661+0000","last_fullsized":"2024-09-13T23:42:59.554661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:59.666873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020617040000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.546758+0000","last_change":"2024-09-13T23:42:59.547128+0000","last_active":"2024-09-13T23:42:59.546758+0000","last_peered":"2024-09-13T23:42:59.546758+0000","last_clean":"2024-09-13T23:42:59.546758+0000","last_became_active":"2024-09-13T23:42:57.541873+0000","last_became_peered":"2024-09-13T23:42:57.541873+0000","last_unstale":"2024-09-13T23:42:59.546758+0000","last_undegraded":"2024-09-13T23:42:59.546758+0000","last_fullsized":"2024-09-13T23:42:59.546758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:32:26.192913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064112499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.554276+0000","last_change":"2024-09-13T23:42:59.554599+0000","last_active":"2024-09-13T23:42:59.554276+0000","last_peered":"2024-09-13T23:42:59.554276+0000","last_clean":"2024-09-13T23:42:59.554276+0000","last_became_active":"2024-09-13T23:42:57.546147+0000","last_became_peered":"2024-09-13T23:42:57.546147+0000","last_unstale":"2024-09-13T23:42:59.554276+0000","last_undegraded":"2024-09-13T23:42:59.554276+0000","last_fullsized":"2024-09-13T23:42:59.554276+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:45.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017185010000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.546683+0000","last_change":"2024-09-13T23:42:59.547142+0000","last_active":"2024-09-13T23:42:59.546683+0000","last_peered":"2024-09-13T23:42:59.546683+0000","last_clean":"2024-09-13T23:42:59.546683+0000","last_became_active":"2024-09-13T23:42:58.198140+0000","last_became_peered":"2024-09-13T23:42:58.198140+0000","last_unstale":"2024-09-13T23:42:59.546683+0000","last_undegraded":"2024-09-13T23:42:59.546683+0000","last_fullsized":"2024-09-13T23:42:59.546683+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:29:12.194907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069103000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.554162+0000","last_change":"2024-09-13T23:42:59.554162+0000","last_active":"2024-09-13T23:42:59.554162+0000","last_peered":"2024-09-13T23:42:59.554162+0000","last_clean":"2024-09-13T23:42:59.554162+0000","last_became_active":"2024-09-13T23:42:57.545386+0000","last_became_peered":"2024-09-13T23:42:57.545386+0000","last_unstale":"2024-09-13T23:42:59.554162+0000","last_undegraded":"2024-09-13T23:42:59.554162+0000","last_fullsized":"2024-09-13T23:42:59.554162+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:14:47.540566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001412859,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.653958+0000","last_change":"2024-09-13T23:42:59.653958+0000","last_active":"2024-09-13T23:42:59.653958+0000","last_peered":"2024-09-13T23:42:59.653958+0000","last_clean":"2024-09-13T23:42:59.653958+0000","last_became_active":"2024-09-13T23:42:58.198220+0000","last_became_peered":"2024-09-13T23:42:58.198220+0000","last_unstale":"2024-09-13T23:42:59.653958+0000","last_undegraded":"2024-09-13T23:42:59.653958+0000","last_fullsized":"2024-09-13T23:42:59.653958+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:52:07.247318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071471200000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.553002+0000","last_change":"2024-09-13T23:42:59.553462+0000","last_active":"2024-09-13T23:42:59.553002+0000","last_peered":"2024-09-13T23:42:59.553002+0000","last_clean":"2024-09-13T23:42:59.553002+0000","last_became_active":"2024-09-13T23:42:57.544086+0000","last_became_peered":"2024-09-13T23:42:57.544086+0000","last_unstale":"2024-09-13T23:42:59.553002+0000","last_undegraded":"2024-09-13T23:42:59.553002+0000","last_fullsized":"2024-09-13T23:42:59.553002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:01.717363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071337400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.554067+0000","last_change":"2024-09-13T23:42:59.554566+0000","last_active":"2024-09-13T23:42:59.554067+0000","last_peered":"2024-09-13T23:42:59.554067+0000","last_clean":"2024-09-13T23:42:59.554067+0000","last_became_active":"2024-09-13T23:42:57.544922+0000","last_became_peered":"2024-09-13T23:42:57.544922+0000","last_unstale":"2024-09-13T23:42:59.554067+0000","last_undegraded":"2024-09-13T23:42:59.554067+0000","last_fullsized":"2024-09-13T23:42:59.554067+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:26.560800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016787670000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T23:42:58.542414+0000","last_change":"2024-09-13T23:42:51.641681+0000","last_active":"2024-09-13T23:42:58.542414+0000","last_peered":"2024-09-13T23:42:58.542414+0000","last_clean":"2024-09-13T23:42:58.542414+0000","last_became_active":"2024-09-13T23:42:51.639169+0000","last_became_peered":"2024-09-13T23:42:51.639169+0000","last_unstale":"2024-09-13T23:42:58.542414+0000","last_undegraded":"2024-09-13T23:42:58.542414+0000","last_fullsized":"2024-09-13T23:42:58.542414+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:50.484454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:50.484454+0000","last_clean_scrub_stamp":"2024-09-13T23:42:50.484454+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:20.736624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":49660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]}]},{"osd":2,"up_from":11,"seq":47244640263,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":745472,"data_stored":639309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":1,"up_from":10,"seq":42949672967,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":753664,"data_stored":640047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":0,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T23:43:02.850 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T23:43:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a1d3e1700 1 Processor -- start 2024-09-13T23:43:02.956 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a1d3e1700 1 -- start start 2024-09-13T23:43:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a1d3e1700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2a1812d0e0 0x7f2a1812d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a1d3e1700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1806f700 con 0x7f2a1812da00 2024-09-13T23:43:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a1d3e1700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a18071390 con 0x7f2a1812d0e0 2024-09-13T23:43:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a17fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f2a1812d0e0 0x7f2a1812d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a177fe700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f2a1812da00 0x7f2a1806d860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48838/0 (socket says 172.21.15.12:48838) 2024-09-13T23:43:02.957 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a177fe700 1 -- 172.21.15.12:0/2349896454 learned_addr learned my addr 172.21.15.12:0/2349896454 (peer_addr_for_me v1:172.21.15.12:0/0) 2024-09-13T23:43:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a17fff700 1 -- 172.21.15.12:0/2349896454 >> v1:172.21.15.12:6789/0 conn(0x7f2a1812da00 legacy=0x7f2a1806d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a17fff700 1 -- 172.21.15.12:0/2349896454 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a18071080 con 0x7f2a1812d0e0 2024-09-13T23:43:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a16ffd700 1 -- 172.21.15.12:0/2349896454 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1617578762 0 0) 0x7f2a1806f700 con 0x7f2a1812da00 2024-09-13T23:43:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a17fff700 1 --2- 172.21.15.12:0/2349896454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1812d0e0 0x7f2a1812d4c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2a08009800 tx=0x7f2a08009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92e0fda196fee27 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a16ffd700 1 -- 172.21.15.12:0/2349896454 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a08004160 con 0x7f2a1812d0e0 2024-09-13T23:43:02.958 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a16ffd700 1 -- 172.21.15.12:0/2349896454 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a080042c0 con 0x7f2a1812d0e0 2024-09-13T23:43:02.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a16ffd700 1 -- 172.21.15.12:0/2349896454 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a08004580 con 0x7f2a1812d0e0 2024-09-13T23:43:02.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2349896454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1812d0e0 msgr2=0x7f2a1812d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:02.959 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.952+0000 7f2a1d3e1700 1 --2- 172.21.15.12:0/2349896454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1812d0e0 0x7f2a1812d4c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2a08009800 tx=0x7f2a08009b10 comp rx=0 tx=0).stop 2024-09-13T23:43:02.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2349896454 shutdown_connections 2024-09-13T23:43:02.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a1d3e1700 1 --2- 172.21.15.12:0/2349896454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1812d0e0 0x7f2a1812d4c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:02.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2349896454 >> 172.21.15.12:0/2349896454 conn(0x7f2a1809cd60 msgr2=0x7f2a1806c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:02.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2349896454 shutdown_connections 2024-09-13T23:43:02.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2349896454 wait complete. 2024-09-13T23:43:02.960 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a1d3e1700 1 Processor -- start 2024-09-13T23:43:02.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a1d3e1700 1 -- start start 2024-09-13T23:43:02.961 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a1d3e1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1812d0e0 0x7f2a1811ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a1d3e1700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1811bed0 con 0x7f2a1812d0e0 2024-09-13T23:43:02.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a17fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1812d0e0 0x7f2a1811ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a17fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1812d0e0 0x7f2a1811ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39644/0 (socket says 172.21.15.12:39644) 2024-09-13T23:43:02.962 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a17fff700 1 -- 172.21.15.12:0/2014291416 learned_addr learned my addr 172.21.15.12:0/2014291416 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:02.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a17fff700 1 -- 172.21.15.12:0/2014291416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a080094b0 con 0x7f2a1812d0e0 2024-09-13T23:43:02.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a17fff700 1 --2- 172.21.15.12:0/2014291416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1812d0e0 0x7f2a1811ad70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a080090b0 tx=0x7f2a08004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a14ff9700 1 -- 172.21.15.12:0/2014291416 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a08003d50 con 0x7f2a1812d0e0 2024-09-13T23:43:02.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a14ff9700 1 -- 172.21.15.12:0/2014291416 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0801f070 con 0x7f2a1812d0e0 2024-09-13T23:43:02.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2014291416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1811e370 con 0x7f2a1812d0e0 2024-09-13T23:43:02.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a14ff9700 1 -- 172.21.15.12:0/2014291416 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2a08011630 con 0x7f2a1812d0e0 2024-09-13T23:43:02.963 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.956+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2014291416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1811c590 con 0x7f2a1812d0e0 2024-09-13T23:43:02.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.960+0000 7f2a14ff9700 1 -- 172.21.15.12:0/2014291416 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f2a08021030 con 0x7f2a1812d0e0 2024-09-13T23:43:02.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.960+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2014291416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1811dea0 con 0x7f2a1812d0e0 2024-09-13T23:43:02.965 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.960+0000 7f2a14ff9700 1 --2- 172.21.15.12:0/2014291416 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f2a040354f0 0x7f2a040379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:02.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.960+0000 7f2a14ff9700 1 -- 172.21.15.12:0/2014291416 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f2a08048e50 con 0x7f2a1812d0e0 2024-09-13T23:43:02.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.960+0000 7f2a177fe700 1 --2- 172.21.15.12:0/2014291416 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f2a040354f0 0x7f2a040379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:02.966 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.960+0000 7f2a177fe700 1 --2- 172.21.15.12:0/2014291416 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f2a040354f0 0x7f2a040379b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c000c00 tx=0x7f2a0c007a20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:02.975 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:02.968+0000 7f2a14ff9700 1 -- 172.21.15.12:0/2014291416 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2a1811dea0 con 0x7f2a1812d0e0 2024-09-13T23:43:03.146 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.140+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2014291416 --> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a1811dea0 con 0x7f2a040354f0 2024-09-13T23:43:03.150 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.144+0000 7f2a14ff9700 1 -- 172.21.15.12:0/2014291416 <== mgr.4099 v2:172.21.15.167:6824/1427656188 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+35026 (secure 0 0 0) 0x7f2a1811dea0 con 0x7f2a040354f0 2024-09-13T23:43:03.150 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:43:03.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.148+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2014291416 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f2a040354f0 msgr2=0x7f2a040379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:03.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.148+0000 7f2a1d3e1700 1 --2- 172.21.15.12:0/2014291416 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f2a040354f0 0x7f2a040379b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c000c00 tx=0x7f2a0c007a20 comp rx=0 tx=0).stop 2024-09-13T23:43:03.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.148+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2014291416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1812d0e0 msgr2=0x7f2a1811ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:03.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.148+0000 7f2a1d3e1700 1 --2- 172.21.15.12:0/2014291416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1812d0e0 0x7f2a1811ad70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a080090b0 tx=0x7f2a08004770 comp rx=0 tx=0).stop 2024-09-13T23:43:03.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.148+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2014291416 shutdown_connections 2024-09-13T23:43:03.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.148+0000 7f2a1d3e1700 1 --2- 172.21.15.12:0/2014291416 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f2a040354f0 0x7f2a040379b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:03.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.148+0000 7f2a1d3e1700 1 --2- 172.21.15.12:0/2014291416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f2a1812d0e0 0x7f2a1811ad70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:03.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.148+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2014291416 >> 172.21.15.12:0/2014291416 conn(0x7f2a1809cd60 msgr2=0x7f2a181b50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:03.154 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.148+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2014291416 shutdown_connections 2024-09-13T23:43:03.155 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.148+0000 7f2a1d3e1700 1 -- 172.21.15.12:0/2014291416 wait complete. 2024-09-13T23:43:03.155 INFO:teuthology.orchestra.run.smithi012.stderr:dumped all 2024-09-13T23:43:03.168 INFO:teuthology.orchestra.run.smithi012.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-13T23:43:01.460082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":118044,"kb_used_data":1772,"kb_used_omap":6,"kb_used_meta":116217,"kb_avail":377369316,"statfs":{"total":386547056640,"available":386426179584,"internally_reserved":0,"allocated":1814528,"data_stored":1378695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6374,"internal_metadata":119007002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.939755"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.554661+0000","last_change":"2024-09-13T23:42:59.554946+0000","last_active":"2024-09-13T23:42:59.554661+0000","last_peered":"2024-09-13T23:42:59.554661+0000","last_clean":"2024-09-13T23:42:59.554661+0000","last_became_active":"2024-09-13T23:42:57.544266+0000","last_became_peered":"2024-09-13T23:42:57.544266+0000","last_unstale":"2024-09-13T23:42:59.554661+0000","last_undegraded":"2024-09-13T23:42:59.554661+0000","last_fullsized":"2024-09-13T23:42:59.554661+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:09:59.666873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020617040000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.546758+0000","last_change":"2024-09-13T23:42:59.547128+0000","last_active":"2024-09-13T23:42:59.546758+0000","last_peered":"2024-09-13T23:42:59.546758+0000","last_clean":"2024-09-13T23:42:59.546758+0000","last_became_active":"2024-09-13T23:42:57.541873+0000","last_became_peered":"2024-09-13T23:42:57.541873+0000","last_unstale":"2024-09-13T23:42:59.546758+0000","last_undegraded":"2024-09-13T23:42:59.546758+0000","last_fullsized":"2024-09-13T23:42:59.546758+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T07:32:26.192913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064112499999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.554276+0000","last_change":"2024-09-13T23:42:59.554599+0000","last_active":"2024-09-13T23:42:59.554276+0000","last_peered":"2024-09-13T23:42:59.554276+0000","last_clean":"2024-09-13T23:42:59.554276+0000","last_became_active":"2024-09-13T23:42:57.546147+0000","last_became_peered":"2024-09-13T23:42:57.546147+0000","last_unstale":"2024-09-13T23:42:59.554276+0000","last_undegraded":"2024-09-13T23:42:59.554276+0000","last_fullsized":"2024-09-13T23:42:59.554276+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:43:45.556321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017185010000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.546683+0000","last_change":"2024-09-13T23:42:59.547142+0000","last_active":"2024-09-13T23:42:59.546683+0000","last_peered":"2024-09-13T23:42:59.546683+0000","last_clean":"2024-09-13T23:42:59.546683+0000","last_became_active":"2024-09-13T23:42:58.198140+0000","last_became_peered":"2024-09-13T23:42:58.198140+0000","last_unstale":"2024-09-13T23:42:59.546683+0000","last_undegraded":"2024-09-13T23:42:59.546683+0000","last_fullsized":"2024-09-13T23:42:59.546683+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T09:29:12.194907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069103000000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"18'2","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.554162+0000","last_change":"2024-09-13T23:42:59.554162+0000","last_active":"2024-09-13T23:42:59.554162+0000","last_peered":"2024-09-13T23:42:59.554162+0000","last_clean":"2024-09-13T23:42:59.554162+0000","last_became_active":"2024-09-13T23:42:57.545386+0000","last_became_peered":"2024-09-13T23:42:57.545386+0000","last_unstale":"2024-09-13T23:42:59.554162+0000","last_undegraded":"2024-09-13T23:42:59.554162+0000","last_fullsized":"2024-09-13T23:42:59.554162+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T08:14:47.540566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001412859,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.653958+0000","last_change":"2024-09-13T23:42:59.653958+0000","last_active":"2024-09-13T23:42:59.653958+0000","last_peered":"2024-09-13T23:42:59.653958+0000","last_clean":"2024-09-13T23:42:59.653958+0000","last_became_active":"2024-09-13T23:42:58.198220+0000","last_became_peered":"2024-09-13T23:42:58.198220+0000","last_unstale":"2024-09-13T23:42:59.653958+0000","last_undegraded":"2024-09-13T23:42:59.653958+0000","last_fullsized":"2024-09-13T23:42:59.653958+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:52:07.247318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071471200000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.553002+0000","last_change":"2024-09-13T23:42:59.553462+0000","last_active":"2024-09-13T23:42:59.553002+0000","last_peered":"2024-09-13T23:42:59.553002+0000","last_clean":"2024-09-13T23:42:59.553002+0000","last_became_active":"2024-09-13T23:42:57.544086+0000","last_became_peered":"2024-09-13T23:42:57.544086+0000","last_unstale":"2024-09-13T23:42:59.553002+0000","last_undegraded":"2024-09-13T23:42:59.553002+0000","last_fullsized":"2024-09-13T23:42:59.553002+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T06:08:01.717363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071337400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"16'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-13T23:42:59.554067+0000","last_change":"2024-09-13T23:42:59.554566+0000","last_active":"2024-09-13T23:42:59.554067+0000","last_peered":"2024-09-13T23:42:59.554067+0000","last_clean":"2024-09-13T23:42:59.554067+0000","last_became_active":"2024-09-13T23:42:57.544922+0000","last_became_peered":"2024-09-13T23:42:57.544922+0000","last_unstale":"2024-09-13T23:42:59.554067+0000","last_undegraded":"2024-09-13T23:42:59.554067+0000","last_fullsized":"2024-09-13T23:42:59.554067+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:56.519160+0000","last_clean_scrub_stamp":"2024-09-13T23:42:56.519160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T10:46:26.560800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016787670000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"14'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T23:42:58.542414+0000","last_change":"2024-09-13T23:42:51.641681+0000","last_active":"2024-09-13T23:42:58.542414+0000","last_peered":"2024-09-13T23:42:58.542414+0000","last_clean":"2024-09-13T23:42:58.542414+0000","last_became_active":"2024-09-13T23:42:51.639169+0000","last_became_peered":"2024-09-13T23:42:51.639169+0000","last_unstale":"2024-09-13T23:42:58.542414+0000","last_undegraded":"2024-09-13T23:42:58.542414+0000","last_fullsized":"2024-09-13T23:42:58.542414+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T23:42:50.484454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T23:42:50.484454+0000","last_clean_scrub_stamp":"2024-09-13T23:42:50.484454+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:21:20.736624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":11,"seq":47244640262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":49660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]}]},{"osd":2,"up_from":11,"seq":47244640263,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":745472,"data_stored":639309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]}]},{"osd":1,"up_from":10,"seq":42949672967,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":753664,"data_stored":640047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":0,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":49679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T23:43:03.169 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T23:43:03.169 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T23:43:03.169 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T23:43:03.170 DEBUG:teuthology.orchestra.run.smithi012:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T23:43:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.272+0000 7f5d959d7700 1 Processor -- start 2024-09-13T23:43:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.272+0000 7f5d959d7700 1 -- start start 2024-09-13T23:43:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.272+0000 7f5d959d7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d9012a0b0 0x7f5d9012a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.272+0000 7f5d959d7700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d90099500 con 0x7f5d9012ad70 2024-09-13T23:43:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.272+0000 7f5d959d7700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d90098c50 con 0x7f5d9012a0b0 2024-09-13T23:43:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d949d5700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d9012a0b0 0x7f5d9012a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d949d5700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f5d9012a0b0 0x7f5d9012a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39662/0 (socket says 172.21.15.12:39662) 2024-09-13T23:43:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d949d5700 1 -- 172.21.15.12:0/1816032276 learned_addr learned my addr 172.21.15.12:0/1816032276 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d8ffff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f5d9012ad70 0x7f5d9012b1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:48852/0 (socket says 172.21.15.12:48852) 2024-09-13T23:43:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d949d5700 1 -- 172.21.15.12:0/1816032276 >> v1:172.21.15.12:6789/0 conn(0x7f5d9012ad70 legacy=0x7f5d9012b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:03.280 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d949d5700 1 -- 172.21.15.12:0/1816032276 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9009ccb0 con 0x7f5d9012a0b0 2024-09-13T23:43:03.281 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d949d5700 1 --2- 172.21.15.12:0/1816032276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d9012a0b0 0x7f5d9012a490 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009890 tx=0x7f5d84009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781d4551b2f28ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d8f7fe700 1 -- 172.21.15.12:0/1816032276 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d84004160 con 0x7f5d9012a0b0 2024-09-13T23:43:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d8f7fe700 1 -- 172.21.15.12:0/1816032276 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d840042c0 con 0x7f5d9012a0b0 2024-09-13T23:43:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d8f7fe700 1 -- 172.21.15.12:0/1816032276 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d84004580 con 0x7f5d9012a0b0 2024-09-13T23:43:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d959d7700 1 -- 172.21.15.12:0/1816032276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d9012a0b0 msgr2=0x7f5d9012a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d959d7700 1 --2- 172.21.15.12:0/1816032276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d9012a0b0 0x7f5d9012a490 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009890 tx=0x7f5d84009ba0 comp rx=0 tx=0).stop 2024-09-13T23:43:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d959d7700 1 -- 172.21.15.12:0/1816032276 shutdown_connections 2024-09-13T23:43:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d959d7700 1 --2- 172.21.15.12:0/1816032276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d9012a0b0 0x7f5d9012a490 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d959d7700 1 -- 172.21.15.12:0/1816032276 >> 172.21.15.12:0/1816032276 conn(0x7f5d9009b620 msgr2=0x7f5d900714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d959d7700 1 -- 172.21.15.12:0/1816032276 shutdown_connections 2024-09-13T23:43:03.282 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d959d7700 1 -- 172.21.15.12:0/1816032276 wait complete. 2024-09-13T23:43:03.283 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d959d7700 1 Processor -- start 2024-09-13T23:43:03.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d959d7700 1 -- start start 2024-09-13T23:43:03.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d959d7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d9012a0b0 0x7f5d9006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:03.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.276+0000 7f5d959d7700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9006e690 con 0x7f5d9012a0b0 2024-09-13T23:43:03.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d949d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d9012a0b0 0x7f5d9006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:03.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d949d5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d9012a0b0 0x7f5d9006d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:39670/0 (socket says 172.21.15.12:39670) 2024-09-13T23:43:03.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d949d5700 1 -- 172.21.15.12:0/2442874984 learned_addr learned my addr 172.21.15.12:0/2442874984 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:43:03.284 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d949d5700 1 -- 172.21.15.12:0/2442874984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d84009570 con 0x7f5d9012a0b0 2024-09-13T23:43:03.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d949d5700 1 --2- 172.21.15.12:0/2442874984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d9012a0b0 0x7f5d9006d530 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009be0 tx=0x7f5d84004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:03.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d8d7fa700 1 -- 172.21.15.12:0/2442874984 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d8400b670 con 0x7f5d9012a0b0 2024-09-13T23:43:03.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d8d7fa700 1 -- 172.21.15.12:0/2442874984 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8400b7d0 con 0x7f5d9012a0b0 2024-09-13T23:43:03.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d8d7fa700 1 -- 172.21.15.12:0/2442874984 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d8400ba70 con 0x7f5d9012a0b0 2024-09-13T23:43:03.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d959d7700 1 -- 172.21.15.12:0/2442874984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9006eda0 con 0x7f5d9012a0b0 2024-09-13T23:43:03.285 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d959d7700 1 -- 172.21.15.12:0/2442874984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9011da60 con 0x7f5d9012a0b0 2024-09-13T23:43:03.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d959d7700 1 -- 172.21.15.12:0/2442874984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9012a490 con 0x7f5d9012a0b0 2024-09-13T23:43:03.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d8d7fa700 1 -- 172.21.15.12:0/2442874984 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f5d84021070 con 0x7f5d9012a0b0 2024-09-13T23:43:03.286 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d8d7fa700 1 --2- 172.21.15.12:0/2442874984 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f5d740354a0 0x7f5d74037960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:03.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d8d7fa700 1 -- 172.21.15.12:0/2442874984 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f5d84048e90 con 0x7f5d9012a0b0 2024-09-13T23:43:03.287 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.280+0000 7f5d8ffff700 1 --2- 172.21.15.12:0/2442874984 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f5d740354a0 0x7f5d74037960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:03.288 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.284+0000 7f5d8ffff700 1 --2- 172.21.15.12:0/2442874984 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f5d740354a0 0x7f5d74037960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c000f60 tx=0x7f5d7c007b30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:03.295 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.288+0000 7f5d8d7fa700 1 -- 172.21.15.12:0/2442874984 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5d8402cdc0 con 0x7f5d9012a0b0 2024-09-13T23:43:03.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.500+0000 7f5d959d7700 1 -- 172.21.15.12:0/2442874984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5d9011dd10 con 0x7f5d9012a0b0 2024-09-13T23:43:03.505 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.500+0000 7f5d8d7fa700 1 -- 172.21.15.12:0/2442874984 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5d8402cbe0 con 0x7f5d9012a0b0 2024-09-13T23:43:03.506 INFO:teuthology.orchestra.run.smithi012.stdout: 2024-09-13T23:43:03.506 INFO:teuthology.orchestra.run.smithi012.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T23:43:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.504+0000 7f5d959d7700 1 -- 172.21.15.12:0/2442874984 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f5d740354a0 msgr2=0x7f5d74037960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.504+0000 7f5d959d7700 1 --2- 172.21.15.12:0/2442874984 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f5d740354a0 0x7f5d74037960 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c000f60 tx=0x7f5d7c007b30 comp rx=0 tx=0).stop 2024-09-13T23:43:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.504+0000 7f5d959d7700 1 -- 172.21.15.12:0/2442874984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d9012a0b0 msgr2=0x7f5d9006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.504+0000 7f5d959d7700 1 --2- 172.21.15.12:0/2442874984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d9012a0b0 0x7f5d9006d530 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009be0 tx=0x7f5d84004770 comp rx=0 tx=0).stop 2024-09-13T23:43:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.504+0000 7f5d959d7700 1 -- 172.21.15.12:0/2442874984 shutdown_connections 2024-09-13T23:43:03.508 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.504+0000 7f5d959d7700 1 --2- 172.21.15.12:0/2442874984 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f5d740354a0 0x7f5d74037960 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:03.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.504+0000 7f5d959d7700 1 --2- 172.21.15.12:0/2442874984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f5d9012a0b0 0x7f5d9006d530 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:03.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.504+0000 7f5d959d7700 1 -- 172.21.15.12:0/2442874984 >> 172.21.15.12:0/2442874984 conn(0x7f5d9009b620 msgr2=0x7f5d9006f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:03.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.504+0000 7f5d959d7700 1 -- 172.21.15.12:0/2442874984 shutdown_connections 2024-09-13T23:43:03.509 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:43:03.504+0000 7f5d959d7700 1 -- 172.21.15.12:0/2442874984 wait complete. 2024-09-13T23:43:03.525 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T23:43:03.525 INFO:teuthology.run_tasks:Running task rgw... 2024-09-13T23:43:03.544 DEBUG:tasks.rgw:config is {'client.0': None} 2024-09-13T23:43:03.545 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-09-13T23:43:03.545 INFO:tasks.rgw:Creating data pools 2024-09-13T23:43:03.545 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-09-13T23:43:03.545 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=3 m=1 name=testprofile --cluster ceph 2024-09-13T23:43:03.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f465920b700 1 Processor -- start 2024-09-13T23:43:03.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f465920b700 1 -- start start 2024-09-13T23:43:03.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f465920b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4654114810 0x7f4654112820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:03.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f465920b700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4654112df0 con 0x7f4654114340 2024-09-13T23:43:03.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f465920b700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4654121550 con 0x7f4654114810 2024-09-13T23:43:03.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f46537fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4654114810 0x7f4654112820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:03.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f46537fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f4654114810 0x7f4654112820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:46450/0 (socket says 172.21.15.167:46450) 2024-09-13T23:43:03.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f46537fe700 1 -- 172.21.15.167:0/3519397857 learned_addr learned my addr 172.21.15.167:0/3519397857 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:03.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f4653fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7f4654114340 0x7f4654112110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:41328/0 (socket says 172.21.15.167:41328) 2024-09-13T23:43:03.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f46537fe700 1 -- 172.21.15.167:0/3519397857 >> v1:172.21.15.12:6789/0 conn(0x7f4654114340 legacy=0x7f4654112110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:03.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f46537fe700 1 -- 172.21.15.167:0/3519397857 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4654121690 con 0x7f4654114810 2024-09-13T23:43:03.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f46537fe700 1 --2- 172.21.15.167:0/3519397857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4654114810 0x7f4654112820 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4644009e50 tx=0x7f464400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98e110f15cf0afb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:03.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f4652ffd700 1 -- 172.21.15.167:0/3519397857 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f464400e070 con 0x7f4654114810 2024-09-13T23:43:03.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f4652ffd700 1 -- 172.21.15.167:0/3519397857 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4644003a40 con 0x7f4654114810 2024-09-13T23:43:03.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f4652ffd700 1 -- 172.21.15.167:0/3519397857 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4644003d00 con 0x7f4654114810 2024-09-13T23:43:03.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f465920b700 1 -- 172.21.15.167:0/3519397857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4654114810 msgr2=0x7f4654112820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:03.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f465920b700 1 --2- 172.21.15.167:0/3519397857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4654114810 0x7f4654112820 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4644009e50 tx=0x7f464400b330 comp rx=0 tx=0).stop 2024-09-13T23:43:03.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f465920b700 1 -- 172.21.15.167:0/3519397857 shutdown_connections 2024-09-13T23:43:03.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f465920b700 1 --2- 172.21.15.167:0/3519397857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4654114810 0x7f4654112820 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:03.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f465920b700 1 -- 172.21.15.167:0/3519397857 >> 172.21.15.167:0/3519397857 conn(0x7f465406e550 msgr2=0x7f4654070970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:03.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f465920b700 1 -- 172.21.15.167:0/3519397857 shutdown_connections 2024-09-13T23:43:03.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.668+0000 7f465920b700 1 -- 172.21.15.167:0/3519397857 wait complete. 2024-09-13T23:43:03.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f465920b700 1 Processor -- start 2024-09-13T23:43:03.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f465920b700 1 -- start start 2024-09-13T23:43:03.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f465920b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4654114340 0x7f46541af170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:03.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f465920b700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46541af740 con 0x7f4654114340 2024-09-13T23:43:03.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f4653fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4654114340 0x7f46541af170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:03.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f4653fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4654114340 0x7f46541af170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:46464/0 (socket says 172.21.15.167:46464) 2024-09-13T23:43:03.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f4653fff700 1 -- 172.21.15.167:0/1344650972 learned_addr learned my addr 172.21.15.167:0/1344650972 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:03.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f4653fff700 1 -- 172.21.15.167:0/1344650972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4644009b30 con 0x7f4654114340 2024-09-13T23:43:03.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f4653fff700 1 --2- 172.21.15.167:0/1344650972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4654114340 0x7f46541af170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f463c00c480 tx=0x7f463c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:03.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f4650ff9700 1 -- 172.21.15.167:0/1344650972 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f463c005c00 con 0x7f4654114340 2024-09-13T23:43:03.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f465920b700 1 -- 172.21.15.167:0/1344650972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46541af940 con 0x7f4654114340 2024-09-13T23:43:03.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f4650ff9700 1 -- 172.21.15.167:0/1344650972 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c005d60 con 0x7f4654114340 2024-09-13T23:43:03.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f4650ff9700 1 -- 172.21.15.167:0/1344650972 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f463c0070d0 con 0x7f4654114340 2024-09-13T23:43:03.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f465920b700 1 -- 172.21.15.167:0/1344650972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46541a9190 con 0x7f4654114340 2024-09-13T23:43:03.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.672+0000 7f465920b700 1 -- 172.21.15.167:0/1344650972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465404f020 con 0x7f4654114340 2024-09-13T23:43:03.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.676+0000 7f4650ff9700 1 -- 172.21.15.167:0/1344650972 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f463c007230 con 0x7f4654114340 2024-09-13T23:43:03.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.676+0000 7f4650ff9700 1 --2- 172.21.15.167:0/1344650972 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f46340354f0 0x7f46340379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:03.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.676+0000 7f4650ff9700 1 -- 172.21.15.167:0/1344650972 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 3538+0+0 (secure 0 0 0) 0x7f463c046b20 con 0x7f4654114340 2024-09-13T23:43:03.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.676+0000 7f46537fe700 1 --2- 172.21.15.167:0/1344650972 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f46340354f0 0x7f46340379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:03.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.676+0000 7f46537fe700 1 --2- 172.21.15.167:0/1344650972 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f46340354f0 0x7f46340379b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4644000f80 tx=0x7f4644009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:03.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.684+0000 7f4650ff9700 1 -- 172.21.15.167:0/1344650972 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f463c019890 con 0x7f4654114340 2024-09-13T23:43:03.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:03.864+0000 7f465920b700 1 -- 172.21.15.167:0/1344650972 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "client.0", "profile": ["crush-failure-domain=osd", "k=3", "m=1", "name=testprofile"]} v 0) v1 -- 0x7f4654111880 con 0x7f4654114340 2024-09-13T23:43:04.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.560+0000 7f4650ff9700 1 -- 172.21.15.167:0/1344650972 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "client.0", "profile": ["crush-failure-domain=osd", "k=3", "m=1", "name=testprofile"]}]=0 v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7f463c017070 con 0x7f4654114340 2024-09-13T23:43:04.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.564+0000 7f465920b700 1 -- 172.21.15.167:0/1344650972 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f46340354f0 msgr2=0x7f46340379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:04.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.564+0000 7f465920b700 1 --2- 172.21.15.167:0/1344650972 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f46340354f0 0x7f46340379b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4644000f80 tx=0x7f4644009000 comp rx=0 tx=0).stop 2024-09-13T23:43:04.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.564+0000 7f465920b700 1 -- 172.21.15.167:0/1344650972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4654114340 msgr2=0x7f46541af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:04.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.564+0000 7f465920b700 1 --2- 172.21.15.167:0/1344650972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4654114340 0x7f46541af170 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f463c00c480 tx=0x7f463c00c840 comp rx=0 tx=0).stop 2024-09-13T23:43:04.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.564+0000 7f465920b700 1 -- 172.21.15.167:0/1344650972 shutdown_connections 2024-09-13T23:43:04.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.564+0000 7f465920b700 1 --2- 172.21.15.167:0/1344650972 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f46340354f0 0x7f46340379b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:04.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.564+0000 7f465920b700 1 --2- 172.21.15.167:0/1344650972 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f4654114340 0x7f46541af170 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:04.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.564+0000 7f465920b700 1 -- 172.21.15.167:0/1344650972 >> 172.21.15.167:0/1344650972 conn(0x7f465406e550 msgr2=0x7f465410fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:04.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.564+0000 7f465920b700 1 -- 172.21.15.167:0/1344650972 shutdown_connections 2024-09-13T23:43:04.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.564+0000 7f465920b700 1 -- 172.21.15.167:0/1344650972 wait complete. 2024-09-13T23:43:04.587 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2024-09-13T23:43:04.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe528d0700 1 Processor -- start 2024-09-13T23:43:04.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe528d0700 1 -- start start 2024-09-13T23:43:04.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe528d0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efe4c10f1f0 0x7efe4c118780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:04.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe528d0700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c118cc0 con 0x7efe4c10e700 2024-09-13T23:43:04.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe528d0700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4c119e60 con 0x7efe4c10f1f0 2024-09-13T23:43:04.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe510cd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7efe4c10f1f0 0x7efe4c118780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:04.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe518ce700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7efe4c10e700 0x7efe4c10eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:41334/0 (socket says 172.21.15.167:41334) 2024-09-13T23:43:04.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe518ce700 1 -- 172.21.15.167:0/1936566860 learned_addr learned my addr 172.21.15.167:0/1936566860 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:04.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe510cd700 1 -- 172.21.15.167:0/1936566860 >> v1:172.21.15.12:6789/0 conn(0x7efe4c10e700 legacy=0x7efe4c10eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:04.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe510cd700 1 -- 172.21.15.167:0/1936566860 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c119fa0 con 0x7efe4c10f1f0 2024-09-13T23:43:04.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe510cd700 1 --2- 172.21.15.167:0/1936566860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c10f1f0 0x7efe4c118780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe480097d0 tx=0x7efe48009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69fef4101ad2a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:04.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe508cc700 1 -- 172.21.15.167:0/1936566860 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe480041c0 con 0x7efe4c10f1f0 2024-09-13T23:43:04.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe508cc700 1 -- 172.21.15.167:0/1936566860 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe48004320 con 0x7efe4c10f1f0 2024-09-13T23:43:04.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe508cc700 1 -- 172.21.15.167:0/1936566860 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe480036c0 con 0x7efe4c10f1f0 2024-09-13T23:43:04.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe528d0700 1 -- 172.21.15.167:0/1936566860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c10f1f0 msgr2=0x7efe4c118780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:04.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe528d0700 1 --2- 172.21.15.167:0/1936566860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c10f1f0 0x7efe4c118780 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe480097d0 tx=0x7efe48009ae0 comp rx=0 tx=0).stop 2024-09-13T23:43:04.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe528d0700 1 -- 172.21.15.167:0/1936566860 shutdown_connections 2024-09-13T23:43:04.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe528d0700 1 --2- 172.21.15.167:0/1936566860 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c10f1f0 0x7efe4c118780 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:04.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe528d0700 1 -- 172.21.15.167:0/1936566860 >> 172.21.15.167:0/1936566860 conn(0x7efe4c06e550 msgr2=0x7efe4c070970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:04.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe528d0700 1 -- 172.21.15.167:0/1936566860 shutdown_connections 2024-09-13T23:43:04.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.700+0000 7efe528d0700 1 -- 172.21.15.167:0/1936566860 wait complete. 2024-09-13T23:43:04.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe528d0700 1 Processor -- start 2024-09-13T23:43:04.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe528d0700 1 -- start start 2024-09-13T23:43:04.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe528d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c10e700 0x7efe4c1a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:04.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe528d0700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c1a2310 con 0x7efe4c10e700 2024-09-13T23:43:04.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe518ce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c10e700 0x7efe4c1a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:04.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe518ce700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c10e700 0x7efe4c1a1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:46482/0 (socket says 172.21.15.167:46482) 2024-09-13T23:43:04.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe518ce700 1 -- 172.21.15.167:0/1436434370 learned_addr learned my addr 172.21.15.167:0/1436434370 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:04.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe518ce700 1 -- 172.21.15.167:0/1436434370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe480094b0 con 0x7efe4c10e700 2024-09-13T23:43:04.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe518ce700 1 --2- 172.21.15.167:0/1436434370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c10e700 0x7efe4c1a1dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00c680 tx=0x7efe3c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:04.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe427fc700 1 -- 172.21.15.167:0/1436434370 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe3c005cc0 con 0x7efe4c10e700 2024-09-13T23:43:04.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe427fc700 1 -- 172.21.15.167:0/1436434370 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c005e20 con 0x7efe4c10e700 2024-09-13T23:43:04.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe427fc700 1 -- 172.21.15.167:0/1436434370 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe3c007200 con 0x7efe4c10e700 2024-09-13T23:43:04.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe528d0700 1 -- 172.21.15.167:0/1436434370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4c1a2510 con 0x7efe4c10e700 2024-09-13T23:43:04.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.704+0000 7efe528d0700 1 -- 172.21.15.167:0/1436434370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c1a64b0 con 0x7efe4c10e700 2024-09-13T23:43:04.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.708+0000 7efe528d0700 1 -- 172.21.15.167:0/1436434370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe4c04f020 con 0x7efe4c10e700 2024-09-13T23:43:04.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.708+0000 7efe427fc700 1 -- 172.21.15.167:0/1436434370 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7efe3c007360 con 0x7efe4c10e700 2024-09-13T23:43:04.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.708+0000 7efe427fc700 1 --2- 172.21.15.167:0/1436434370 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7efe30037700 0x7efe30039bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:04.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.708+0000 7efe427fc700 1 -- 172.21.15.167:0/1436434370 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3753+0+0 (secure 0 0 0) 0x7efe3c046c40 con 0x7efe4c10e700 2024-09-13T23:43:04.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.708+0000 7efe510cd700 1 --2- 172.21.15.167:0/1436434370 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7efe30037700 0x7efe30039bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:04.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.708+0000 7efe510cd700 1 --2- 172.21.15.167:0/1436434370 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7efe30037700 0x7efe30039bc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efe48014040 tx=0x7efe48010000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:04.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.716+0000 7efe427fc700 1 -- 172.21.15.167:0/1436434370 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efe3c01b3b0 con 0x7efe4c10e700 2024-09-13T23:43:04.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:04.888+0000 7efe528d0700 1 -- 172.21.15.167:0/1436434370 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64, "pool_type": "erasure", "erasure_code_profile": "client.0"} v 0) v1 -- 0x7efe4c119840 con 0x7efe4c10e700 2024-09-13T23:43:06.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.011+0000 7efe427fc700 1 -- 172.21.15.167:0/1436434370 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64, "pool_type": "erasure", "erasure_code_profile": "client.0"}]=0 pool 'default.rgw.buckets.data' created v21) v1 ==== 227+0+0 (secure 0 0 0) 0x7efe3c017070 con 0x7efe4c10e700 2024-09-13T23:43:06.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.023+0000 7efe528d0700 1 -- 172.21.15.167:0/1436434370 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7efe30037700 msgr2=0x7efe30039bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:06.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.023+0000 7efe528d0700 1 --2- 172.21.15.167:0/1436434370 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7efe30037700 0x7efe30039bc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efe48014040 tx=0x7efe48010000 comp rx=0 tx=0).stop 2024-09-13T23:43:06.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.023+0000 7efe528d0700 1 -- 172.21.15.167:0/1436434370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c10e700 msgr2=0x7efe4c1a1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:06.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.023+0000 7efe528d0700 1 --2- 172.21.15.167:0/1436434370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c10e700 0x7efe4c1a1dd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efe3c00c680 tx=0x7efe3c00ca40 comp rx=0 tx=0).stop 2024-09-13T23:43:06.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.023+0000 7efe528d0700 1 -- 172.21.15.167:0/1436434370 shutdown_connections 2024-09-13T23:43:06.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.023+0000 7efe528d0700 1 --2- 172.21.15.167:0/1436434370 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7efe30037700 0x7efe30039bc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:06.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.023+0000 7efe528d0700 1 --2- 172.21.15.167:0/1436434370 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7efe4c10e700 0x7efe4c1a1dd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:06.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.023+0000 7efe528d0700 1 -- 172.21.15.167:0/1436434370 >> 172.21.15.167:0/1436434370 conn(0x7efe4c06e550 msgr2=0x7efe4c115a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.023+0000 7efe528d0700 1 -- 172.21.15.167:0/1436434370 shutdown_connections 2024-09-13T23:43:06.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.023+0000 7efe528d0700 1 -- 172.21.15.167:0/1436434370 wait complete. 2024-09-13T23:43:06.038 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'default.rgw.buckets.data' created 2024-09-13T23:43:06.051 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-09-13T23:43:06.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.151+0000 7fc54ffff700 1 Processor -- start 2024-09-13T23:43:06.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.151+0000 7fc54ffff700 1 -- start start 2024-09-13T23:43:06.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.151+0000 7fc54ffff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc55006ad30 0x7fc55006b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:06.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.151+0000 7fc54ffff700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55011bcd0 con 0x7fc55006b740 2024-09-13T23:43:06.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.151+0000 7fc54ffff700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55011ce70 con 0x7fc55006ad30 2024-09-13T23:43:06.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.151+0000 7fc54effd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc55006ad30 0x7fc55006b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:06.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.151+0000 7fc54e7fc700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fc55006b740 0x7fc55011a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:41336/0 (socket says 172.21.15.167:41336) 2024-09-13T23:43:06.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.151+0000 7fc54e7fc700 1 -- 172.21.15.167:0/2770334636 learned_addr learned my addr 172.21.15.167:0/2770334636 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:06.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.151+0000 7fc54effd700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fc55006ad30 0x7fc55006b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:46486/0 (socket says 172.21.15.167:46486) 2024-09-13T23:43:06.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.151+0000 7fc54effd700 1 -- 172.21.15.167:0/2770334636 >> v1:172.21.15.12:6789/0 conn(0x7fc55006b740 legacy=0x7fc55011a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:06.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.151+0000 7fc54effd700 1 -- 172.21.15.167:0/2770334636 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55011cfb0 con 0x7fc55006ad30 2024-09-13T23:43:06.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.151+0000 7fc54effd700 1 --2- 172.21.15.167:0/2770334636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55006ad30 0x7fc55006b110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc5440097d0 tx=0x7fc544009ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b013a6c7f4094f9d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:06.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54dffb700 1 -- 172.21.15.167:0/2770334636 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5440041c0 con 0x7fc55006ad30 2024-09-13T23:43:06.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54dffb700 1 -- 172.21.15.167:0/2770334636 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc544004320 con 0x7fc55006ad30 2024-09-13T23:43:06.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54dffb700 1 -- 172.21.15.167:0/2770334636 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5440036c0 con 0x7fc55006ad30 2024-09-13T23:43:06.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54ffff700 1 -- 172.21.15.167:0/2770334636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55006ad30 msgr2=0x7fc55006b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:06.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54ffff700 1 --2- 172.21.15.167:0/2770334636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55006ad30 0x7fc55006b110 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc5440097d0 tx=0x7fc544009ae0 comp rx=0 tx=0).stop 2024-09-13T23:43:06.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54ffff700 1 -- 172.21.15.167:0/2770334636 shutdown_connections 2024-09-13T23:43:06.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54ffff700 1 --2- 172.21.15.167:0/2770334636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55006ad30 0x7fc55006b110 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:06.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54ffff700 1 -- 172.21.15.167:0/2770334636 >> 172.21.15.167:0/2770334636 conn(0x7fc55006ed40 msgr2=0x7fc550071160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:06.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54ffff700 1 -- 172.21.15.167:0/2770334636 shutdown_connections 2024-09-13T23:43:06.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54ffff700 1 -- 172.21.15.167:0/2770334636 wait complete. 2024-09-13T23:43:06.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54ffff700 1 Processor -- start 2024-09-13T23:43:06.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54ffff700 1 -- start start 2024-09-13T23:43:06.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54ffff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55006ad30 0x7fc5501aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:06.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54ffff700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5501ab1b0 con 0x7fc55006ad30 2024-09-13T23:43:06.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55006ad30 0x7fc5501aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:06.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.155+0000 7fc54effd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55006ad30 0x7fc5501aabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:46494/0 (socket says 172.21.15.167:46494) 2024-09-13T23:43:06.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc54effd700 1 -- 172.21.15.167:0/3627546919 learned_addr learned my addr 172.21.15.167:0/3627546919 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:06.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc54effd700 1 -- 172.21.15.167:0/3627546919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5440094b0 con 0x7fc55006ad30 2024-09-13T23:43:06.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc54effd700 1 --2- 172.21.15.167:0/3627546919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55006ad30 0x7fc5501aabe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc544009b20 tx=0x7fc544004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:06.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc537fff700 1 -- 172.21.15.167:0/3627546919 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc544006ea0 con 0x7fc55006ad30 2024-09-13T23:43:06.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc537fff700 1 -- 172.21.15.167:0/3627546919 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc544011760 con 0x7fc55006ad30 2024-09-13T23:43:06.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc54ffff700 1 -- 172.21.15.167:0/3627546919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5501ab350 con 0x7fc55006ad30 2024-09-13T23:43:06.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc537fff700 1 -- 172.21.15.167:0/3627546919 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc544011a00 con 0x7fc55006ad30 2024-09-13T23:43:06.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc54ffff700 1 -- 172.21.15.167:0/3627546919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5501a4c00 con 0x7fc55006ad30 2024-09-13T23:43:06.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc537fff700 1 -- 172.21.15.167:0/3627546919 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fc544018050 con 0x7fc55006ad30 2024-09-13T23:43:06.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc54ffff700 1 -- 172.21.15.167:0/3627546919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55006b110 con 0x7fc55006ad30 2024-09-13T23:43:06.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc537fff700 1 --2- 172.21.15.167:0/3627546919 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fc53c0354f0 0x7fc53c0379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:06.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc537fff700 1 -- 172.21.15.167:0/3627546919 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4200+0+0 (secure 0 0 0) 0x7fc544048840 con 0x7fc55006ad30 2024-09-13T23:43:06.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.159+0000 7fc54e7fc700 1 --2- 172.21.15.167:0/3627546919 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fc53c0354f0 0x7fc53c0379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:06.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.163+0000 7fc54e7fc700 1 --2- 172.21.15.167:0/3627546919 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fc53c0354f0 0x7fc53c0379b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc5400016a0 tx=0x7fc540007b70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:06.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.171+0000 7fc537fff700 1 -- 172.21.15.167:0/3627546919 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc5440257c0 con 0x7fc55006ad30 2024-09-13T23:43:06.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:06.347+0000 7fc54ffff700 1 -- 172.21.15.167:0/3627546919 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"} v 0) v1 -- 0x7fc55006b740 con 0x7fc55006ad30 2024-09-13T23:43:07.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.015+0000 7fc537fff700 1 -- 172.21.15.167:0/3627546919 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.data' v22) v1 ==== 185+0+0 (secure 0 0 0) 0x7fc544016070 con 0x7fc55006ad30 2024-09-13T23:43:07.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.027+0000 7fc54ffff700 1 -- 172.21.15.167:0/3627546919 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fc53c0354f0 msgr2=0x7fc53c0379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:07.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.027+0000 7fc54ffff700 1 --2- 172.21.15.167:0/3627546919 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fc53c0354f0 0x7fc53c0379b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc5400016a0 tx=0x7fc540007b70 comp rx=0 tx=0).stop 2024-09-13T23:43:07.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.027+0000 7fc54ffff700 1 -- 172.21.15.167:0/3627546919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55006ad30 msgr2=0x7fc5501aabe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:07.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.027+0000 7fc54ffff700 1 --2- 172.21.15.167:0/3627546919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55006ad30 0x7fc5501aabe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc544009b20 tx=0x7fc544004770 comp rx=0 tx=0).stop 2024-09-13T23:43:07.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.031+0000 7fc54ffff700 1 -- 172.21.15.167:0/3627546919 shutdown_connections 2024-09-13T23:43:07.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.031+0000 7fc54ffff700 1 --2- 172.21.15.167:0/3627546919 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fc53c0354f0 0x7fc53c0379b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:07.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.031+0000 7fc54ffff700 1 --2- 172.21.15.167:0/3627546919 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fc55006ad30 0x7fc5501aabe0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:07.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.031+0000 7fc54ffff700 1 -- 172.21.15.167:0/3627546919 >> 172.21.15.167:0/3627546919 conn(0x7fc55006ed40 msgr2=0x7fc550117760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:07.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.039+0000 7fc54ffff700 1 -- 172.21.15.167:0/3627546919 shutdown_connections 2024-09-13T23:43:07.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.039+0000 7fc54ffff700 1 -- 172.21.15.167:0/3627546919 wait complete. 2024-09-13T23:43:07.053 INFO:teuthology.orchestra.run.smithi167.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-09-13T23:43:07.075 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-09-13T23:43:07.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c83fff700 1 Processor -- start 2024-09-13T23:43:07.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c83fff700 1 -- start start 2024-09-13T23:43:07.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c83fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3c84116030 0x7f3c84118420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:07.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c83fff700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c841189f0 con 0x7f3c841132a0 2024-09-13T23:43:07.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c83fff700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c84119b90 con 0x7f3c84116030 2024-09-13T23:43:07.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c827fc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3c84116030 0x7f3c84118420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:07.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c827fc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7f3c84116030 0x7f3c84118420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:48478/0 (socket says 172.21.15.167:48478) 2024-09-13T23:43:07.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c827fc700 1 -- 172.21.15.167:0/2375528770 learned_addr learned my addr 172.21.15.167:0/2375528770 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:07.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c82ffd700 1 --1- 172.21.15.167:0/2375528770 >> v1:172.21.15.12:6789/0 conn(0x7f3c841132a0 0x7f3c84113680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:41812/0 (socket says 172.21.15.167:41812) 2024-09-13T23:43:07.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c827fc700 1 -- 172.21.15.167:0/2375528770 >> v1:172.21.15.12:6789/0 conn(0x7f3c841132a0 legacy=0x7f3c84113680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:07.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c827fc700 1 -- 172.21.15.167:0/2375528770 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c84119cd0 con 0x7f3c84116030 2024-09-13T23:43:07.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c827fc700 1 --2- 172.21.15.167:0/2375528770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c84116030 0x7f3c84118420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009890 tx=0x7f3c74009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8f47f2273f5861 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:07.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c81ffb700 1 -- 172.21.15.167:0/2375528770 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c74004160 con 0x7f3c84116030 2024-09-13T23:43:07.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c81ffb700 1 -- 172.21.15.167:0/2375528770 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c740042c0 con 0x7f3c84116030 2024-09-13T23:43:07.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c81ffb700 1 -- 172.21.15.167:0/2375528770 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c74004580 con 0x7f3c84116030 2024-09-13T23:43:07.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.175+0000 7f3c83fff700 1 -- 172.21.15.167:0/2375528770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c84116030 msgr2=0x7f3c84118420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:07.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c83fff700 1 --2- 172.21.15.167:0/2375528770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c84116030 0x7f3c84118420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009890 tx=0x7f3c74009ba0 comp rx=0 tx=0).stop 2024-09-13T23:43:07.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c83fff700 1 -- 172.21.15.167:0/2375528770 shutdown_connections 2024-09-13T23:43:07.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c83fff700 1 --2- 172.21.15.167:0/2375528770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c84116030 0x7f3c84118420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:07.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c83fff700 1 -- 172.21.15.167:0/2375528770 >> 172.21.15.167:0/2375528770 conn(0x7f3c8406eda0 msgr2=0x7f3c840711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:07.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c83fff700 1 -- 172.21.15.167:0/2375528770 shutdown_connections 2024-09-13T23:43:07.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c83fff700 1 -- 172.21.15.167:0/2375528770 wait complete. 2024-09-13T23:43:07.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c83fff700 1 Processor -- start 2024-09-13T23:43:07.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c83fff700 1 -- start start 2024-09-13T23:43:07.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c83fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c841132a0 0x7f3c841aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:07.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c83fff700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c841ab120 con 0x7f3c841132a0 2024-09-13T23:43:07.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c82ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c841132a0 0x7f3c841aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:07.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c82ffd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c841132a0 0x7f3c841aab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:48488/0 (socket says 172.21.15.167:48488) 2024-09-13T23:43:07.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c82ffd700 1 -- 172.21.15.167:0/1561836966 learned_addr learned my addr 172.21.15.167:0/1561836966 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:07.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c82ffd700 1 -- 172.21.15.167:0/1561836966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c74009570 con 0x7f3c841132a0 2024-09-13T23:43:07.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c82ffd700 1 --2- 172.21.15.167:0/1561836966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c841132a0 0x7f3c841aab50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00c680 tx=0x7f3c6c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:07.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c7b7fe700 1 -- 172.21.15.167:0/1561836966 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c6c005c00 con 0x7f3c841132a0 2024-09-13T23:43:07.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c83fff700 1 -- 172.21.15.167:0/1561836966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c841ab320 con 0x7f3c841132a0 2024-09-13T23:43:07.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c7b7fe700 1 -- 172.21.15.167:0/1561836966 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c6c005d60 con 0x7f3c841132a0 2024-09-13T23:43:07.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c7b7fe700 1 -- 172.21.15.167:0/1561836966 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c6c0070d0 con 0x7f3c841132a0 2024-09-13T23:43:07.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.179+0000 7f3c83fff700 1 -- 172.21.15.167:0/1561836966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c841a4bf0 con 0x7f3c841132a0 2024-09-13T23:43:07.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.183+0000 7f3c7b7fe700 1 -- 172.21.15.167:0/1561836966 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f3c6c007230 con 0x7f3c841132a0 2024-09-13T23:43:07.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.183+0000 7f3c83fff700 1 -- 172.21.15.167:0/1561836966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c84119660 con 0x7f3c841132a0 2024-09-13T23:43:07.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.183+0000 7f3c7b7fe700 1 --2- 172.21.15.167:0/1561836966 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f3c640354a0 0x7f3c64037960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:07.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.183+0000 7f3c7b7fe700 1 -- 172.21.15.167:0/1561836966 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4211+0+0 (secure 0 0 0) 0x7f3c6c046ad0 con 0x7f3c841132a0 2024-09-13T23:43:07.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.183+0000 7f3c827fc700 1 --2- 172.21.15.167:0/1561836966 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f3c640354a0 0x7f3c64037960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:07.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.183+0000 7f3c827fc700 1 --2- 172.21.15.167:0/1561836966 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f3c640354a0 0x7f3c64037960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009be0 tx=0x7f3c74008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:07.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.191+0000 7f3c7b7fe700 1 -- 172.21.15.167:0/1561836966 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3c6c012400 con 0x7f3c841132a0 2024-09-13T23:43:07.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:07.363+0000 7f3c83fff700 1 -- 172.21.15.167:0/1561836966 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64} v 0) v1 -- 0x7f3c84116030 con 0x7f3c841132a0 2024-09-13T23:43:08.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.027+0000 7f3c7b7fe700 1 -- 172.21.15.167:0/1561836966 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.index' created v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3c6c017030 con 0x7f3c841132a0 2024-09-13T23:43:08.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.039+0000 7f3c83fff700 1 -- 172.21.15.167:0/1561836966 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f3c640354a0 msgr2=0x7f3c64037960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:08.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.039+0000 7f3c83fff700 1 --2- 172.21.15.167:0/1561836966 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f3c640354a0 0x7f3c64037960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3c74009be0 tx=0x7f3c74008030 comp rx=0 tx=0).stop 2024-09-13T23:43:08.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.039+0000 7f3c83fff700 1 -- 172.21.15.167:0/1561836966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c841132a0 msgr2=0x7f3c841aab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:08.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.039+0000 7f3c83fff700 1 --2- 172.21.15.167:0/1561836966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c841132a0 0x7f3c841aab50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00c680 tx=0x7f3c6c00ca40 comp rx=0 tx=0).stop 2024-09-13T23:43:08.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.039+0000 7f3c83fff700 1 -- 172.21.15.167:0/1561836966 shutdown_connections 2024-09-13T23:43:08.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.039+0000 7f3c83fff700 1 --2- 172.21.15.167:0/1561836966 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7f3c640354a0 0x7f3c64037960 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:08.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.039+0000 7f3c83fff700 1 --2- 172.21.15.167:0/1561836966 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7f3c841132a0 0x7f3c841aab50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:08.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.039+0000 7f3c83fff700 1 -- 172.21.15.167:0/1561836966 >> 172.21.15.167:0/1561836966 conn(0x7f3c8406eda0 msgr2=0x7f3c8406ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:08.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.039+0000 7f3c83fff700 1 -- 172.21.15.167:0/1561836966 shutdown_connections 2024-09-13T23:43:08.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.039+0000 7f3c83fff700 1 -- 172.21.15.167:0/1561836966 wait complete. 2024-09-13T23:43:08.050 INFO:teuthology.orchestra.run.smithi167.stderr:pool 'default.rgw.buckets.index' created 2024-09-13T23:43:08.066 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-09-13T23:43:08.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd074aaa700 1 Processor -- start 2024-09-13T23:43:08.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd074aaa700 1 -- start start 2024-09-13T23:43:08.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd074aaa700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd070119390 0x7fd07011b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:08.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd074aaa700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07011bd50 con 0x7fd0701166a0 2024-09-13T23:43:08.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd074aaa700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd07011cef0 con 0x7fd070119390 2024-09-13T23:43:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd067fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd070119390 0x7fd07011b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd067fff700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fd070119390 0x7fd07011b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:48490/0 (socket says 172.21.15.167:48490) 2024-09-13T23:43:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd067fff700 1 -- 172.21.15.167:0/1618369603 learned_addr learned my addr 172.21.15.167:0/1618369603 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd06f7fe700 1 --1- 172.21.15.167:0/1618369603 >> v1:172.21.15.12:6789/0 conn(0x7fd0701166a0 0x7fd070116a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:41816/0 (socket says 172.21.15.167:41816) 2024-09-13T23:43:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd067fff700 1 -- 172.21.15.167:0/1618369603 >> v1:172.21.15.12:6789/0 conn(0x7fd0701166a0 legacy=0x7fd070116a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:08.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd067fff700 1 -- 172.21.15.167:0/1618369603 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07011d030 con 0x7fd070119390 2024-09-13T23:43:08.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd067fff700 1 --2- 172.21.15.167:0/1618369603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd070119390 0x7fd07011b780 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd058009890 tx=0x7fd058009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f145352a2839a5c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:08.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd06effd700 1 -- 172.21.15.167:0/1618369603 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd058004160 con 0x7fd070119390 2024-09-13T23:43:08.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd06effd700 1 -- 172.21.15.167:0/1618369603 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0580042c0 con 0x7fd070119390 2024-09-13T23:43:08.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd06effd700 1 -- 172.21.15.167:0/1618369603 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd058004580 con 0x7fd070119390 2024-09-13T23:43:08.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd074aaa700 1 -- 172.21.15.167:0/1618369603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd070119390 msgr2=0x7fd07011b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:08.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd074aaa700 1 --2- 172.21.15.167:0/1618369603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd070119390 0x7fd07011b780 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd058009890 tx=0x7fd058009ba0 comp rx=0 tx=0).stop 2024-09-13T23:43:08.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd074aaa700 1 -- 172.21.15.167:0/1618369603 shutdown_connections 2024-09-13T23:43:08.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd074aaa700 1 --2- 172.21.15.167:0/1618369603 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd070119390 0x7fd07011b780 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:08.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd074aaa700 1 -- 172.21.15.167:0/1618369603 >> 172.21.15.167:0/1618369603 conn(0x7fd07006d4f0 msgr2=0x7fd07010ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:08.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd074aaa700 1 -- 172.21.15.167:0/1618369603 shutdown_connections 2024-09-13T23:43:08.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.167+0000 7fd074aaa700 1 -- 172.21.15.167:0/1618369603 wait complete. 2024-09-13T23:43:08.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd074aaa700 1 Processor -- start 2024-09-13T23:43:08.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd074aaa700 1 -- start start 2024-09-13T23:43:08.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd074aaa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0701166a0 0x7fd0701aac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:08.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd074aaa700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0701ab1e0 con 0x7fd0701166a0 2024-09-13T23:43:08.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd06f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0701166a0 0x7fd0701aac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:08.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd06f7fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0701166a0 0x7fd0701aac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:48502/0 (socket says 172.21.15.167:48502) 2024-09-13T23:43:08.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd06f7fe700 1 -- 172.21.15.167:0/3525522699 learned_addr learned my addr 172.21.15.167:0/3525522699 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:08.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd06f7fe700 1 -- 172.21.15.167:0/3525522699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd058009570 con 0x7fd0701166a0 2024-09-13T23:43:08.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd06f7fe700 1 --2- 172.21.15.167:0/3525522699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0701166a0 0x7fd0701aac10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd06000c480 tx=0x7fd06000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:08.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd06cff9700 1 -- 172.21.15.167:0/3525522699 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd060005c00 con 0x7fd0701166a0 2024-09-13T23:43:08.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd06cff9700 1 -- 172.21.15.167:0/3525522699 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd060005d60 con 0x7fd0701166a0 2024-09-13T23:43:08.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd06cff9700 1 -- 172.21.15.167:0/3525522699 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0600070d0 con 0x7fd0701166a0 2024-09-13T23:43:08.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd074aaa700 1 -- 172.21.15.167:0/3525522699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0701ab3e0 con 0x7fd0701166a0 2024-09-13T23:43:08.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd074aaa700 1 -- 172.21.15.167:0/3525522699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0701a4c80 con 0x7fd0701166a0 2024-09-13T23:43:08.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.171+0000 7fd074aaa700 1 -- 172.21.15.167:0/3525522699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07011c910 con 0x7fd0701166a0 2024-09-13T23:43:08.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.175+0000 7fd06cff9700 1 -- 172.21.15.167:0/3525522699 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fd060007230 con 0x7fd0701166a0 2024-09-13T23:43:08.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.175+0000 7fd06cff9700 1 --2- 172.21.15.167:0/3525522699 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fd0500354f0 0x7fd0500379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:08.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.175+0000 7fd06cff9700 1 -- 172.21.15.167:0/3525522699 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4575+0+0 (secure 0 0 0) 0x7fd060046ea0 con 0x7fd0701166a0 2024-09-13T23:43:08.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.175+0000 7fd067fff700 1 --2- 172.21.15.167:0/3525522699 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fd0500354f0 0x7fd0500379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:08.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.175+0000 7fd067fff700 1 --2- 172.21.15.167:0/3525522699 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fd0500354f0 0x7fd0500379b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd070116400 tx=0x7fd058008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:08.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.183+0000 7fd06cff9700 1 -- 172.21.15.167:0/3525522699 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd060019890 con 0x7fd0701166a0 2024-09-13T23:43:08.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:08.351+0000 7fd074aaa700 1 -- 172.21.15.167:0/3525522699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"} v 0) v1 -- 0x7fd070119390 con 0x7fd0701166a0 2024-09-13T23:43:09.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:09.035+0000 7fd06cff9700 1 -- 172.21.15.167:0/3525522699 <== mon.0 v2:172.21.15.12:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.index' v24) v1 ==== 187+0+0 (secure 0 0 0) 0x7fd060017070 con 0x7fd0701166a0 2024-09-13T23:43:09.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:09.043+0000 7fd074aaa700 1 -- 172.21.15.167:0/3525522699 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fd0500354f0 msgr2=0x7fd0500379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:09.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:09.043+0000 7fd074aaa700 1 --2- 172.21.15.167:0/3525522699 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fd0500354f0 0x7fd0500379b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd070116400 tx=0x7fd058008030 comp rx=0 tx=0).stop 2024-09-13T23:43:09.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:09.043+0000 7fd074aaa700 1 -- 172.21.15.167:0/3525522699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0701166a0 msgr2=0x7fd0701aac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:09.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:09.043+0000 7fd074aaa700 1 --2- 172.21.15.167:0/3525522699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0701166a0 0x7fd0701aac10 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd06000c480 tx=0x7fd06000c840 comp rx=0 tx=0).stop 2024-09-13T23:43:09.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:09.043+0000 7fd074aaa700 1 -- 172.21.15.167:0/3525522699 shutdown_connections 2024-09-13T23:43:09.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:09.043+0000 7fd074aaa700 1 --2- 172.21.15.167:0/3525522699 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fd0500354f0 0x7fd0500379b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:09.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:09.043+0000 7fd074aaa700 1 --2- 172.21.15.167:0/3525522699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fd0701166a0 0x7fd0701aac10 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:09.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:09.043+0000 7fd074aaa700 1 -- 172.21.15.167:0/3525522699 >> 172.21.15.167:0/3525522699 conn(0x7fd07006d4f0 msgr2=0x7fd07011a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:09.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:09.043+0000 7fd074aaa700 1 -- 172.21.15.167:0/3525522699 shutdown_connections 2024-09-13T23:43:09.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:09.043+0000 7fd074aaa700 1 -- 172.21.15.167:0/3525522699 wait complete. 2024-09-13T23:43:09.064 INFO:teuthology.orchestra.run.smithi167.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-09-13T23:43:09.082 DEBUG:tasks.rgw:Pools created 2024-09-13T23:43:09.082 INFO:tasks.rgw:Starting rgw... 2024-09-13T23:43:09.082 INFO:tasks.rgw:rgw client.0 config is {} 2024-09-13T23:43:09.082 INFO:tasks.rgw:Using beast as radosgw frontend 2024-09-13T23:43:09.082 DEBUG:teuthology.orchestra.run.smithi167:> sudo echo -n http://smithi167.front.sepia.ceph.com:80 | sudo tee /home/ubuntu/cephtest/url_file 2024-09-13T23:43:09.092 INFO:teuthology.orchestra.run.smithi167.stdout:http://smithi167.front.sepia.ceph.com:80 2024-09-13T23:43:09.093 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-09-13T23:43:09.103 INFO:tasks.rgw.client.0:Restarting daemon 2024-09-13T23:43:09.103 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-09-13T23:43:09.151 INFO:tasks.rgw.client.0:Started 2024-09-13T23:43:09.152 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi167.front.sepia.ceph.com:80/ 2024-09-13T23:43:09.152 DEBUG:teuthology.orchestra.run.smithi167:> curl http://smithi167.front.sepia.ceph.com:80/ 2024-09-13T23:43:09.191 INFO:teuthology.orchestra.run.smithi167.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-09-13T23:43:09.191 INFO:teuthology.orchestra.run.smithi167.stderr: Dload Upload Total Spent Left Speed 2024-09-13T23:43:09.193 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-09-13T23:43:09.193 INFO:teuthology.orchestra.run.smithi167.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-09-13T23:43:09.193 INFO:teuthology.orchestra.run.smithi167.stderr:curl: (7) Failed to connect to smithi167.front.sepia.ceph.com port 80: Connection refused 2024-09-13T23:43:10.194 DEBUG:teuthology.orchestra.run.smithi167:> curl http://smithi167.front.sepia.ceph.com:80/ 2024-09-13T23:43:10.210 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-09-13T23:43:10.211 INFO:teuthology.orchestra.run.smithi167.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-09-13T23:43:10.211 INFO:teuthology.orchestra.run.smithi167.stderr: Dload Upload Total Spent Left Speed 2024-09-13T23:43:10.211 INFO:teuthology.orchestra.run.smithi167.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-09-13T23:43:10.211 INFO:teuthology.orchestra.run.smithi167.stderr:curl: (7) Failed to connect to smithi167.front.sepia.ceph.com port 80: Connection refused 2024-09-13T23:43:12.212 DEBUG:teuthology.orchestra.run.smithi167:> curl http://smithi167.front.sepia.ceph.com:80/ 2024-09-13T23:43:12.229 INFO:teuthology.orchestra.run.smithi167.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-09-13T23:43:12.230 INFO:teuthology.orchestra.run.smithi167.stderr: Dload Upload Total Spent Left Speed 2024-09-13T23:43:12.230 INFO:teuthology.orchestra.run.smithi167.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-09-13T23:43:12.230 INFO:teuthology.orchestra.run.smithi167.stderr:curl: (7) Failed to connect to smithi167.front.sepia.ceph.com port 80: Connection refused 2024-09-13T23:43:12.230 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-09-13T23:43:16.232 DEBUG:teuthology.orchestra.run.smithi167:> curl http://smithi167.front.sepia.ceph.com:80/ 2024-09-13T23:43:16.250 INFO:teuthology.orchestra.run.smithi167.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-09-13T23:43:16.250 INFO:teuthology.orchestra.run.smithi167.stderr: Dload Upload Total Spent Left Speed 2024-09-13T23:43:16.252 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-09-13T23:43:16.252 INFO:teuthology.orchestra.run.smithi167.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-09-13T23:43:16.252 INFO:teuthology.orchestra.run.smithi167.stderr:curl: (7) Failed to connect to smithi167.front.sepia.ceph.com port 80: Connection refused 2024-09-13T23:43:24.253 DEBUG:teuthology.orchestra.run.smithi167:> curl http://smithi167.front.sepia.ceph.com:80/ 2024-09-13T23:43:24.269 INFO:teuthology.orchestra.run.smithi167.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-09-13T23:43:24.270 INFO:teuthology.orchestra.run.smithi167.stderr: Dload Upload Total Spent Left Speed 2024-09-13T23:43:24.273 INFO:teuthology.orchestra.run.smithi167.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 71333 0 --:--:-- --:--:-- --:--:-- 71333 2024-09-13T23:43:24.274 INFO:teuthology.orchestra.run.smithi167.stdout:anonymous 2024-09-13T23:43:24.274 INFO:teuthology.run_tasks:Running task radosgw-admin... 2024-09-13T23:43:24.355 DEBUG:tasks.radosgw_admin:config is: {'client.0': None} 2024-09-13T23:43:24.356 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-09-13T23:43:24.356 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-09-13T23:43:24.356 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-09-13T23:43:24.397 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:24.398 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:24.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d50c6ef40 1 Processor -- start 2024-09-13T23:43:24.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d50c6ef40 1 -- start start 2024-09-13T23:43:24.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d50c6ef40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x561d9446f000 0x561d94754580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d50c6ef40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561d94533500 con 0x561d9446f400 2024-09-13T23:43:24.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d50c6ef40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x561d944fca00 con 0x561d9446f000 2024-09-13T23:43:24.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d4f427700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x561d9446f000 0x561d94754580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d4ec26700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x561d9446f400 0x561d9456e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:40284/0 (socket says 172.21.15.167:40284) 2024-09-13T23:43:24.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d4ec26700 1 -- 172.21.15.167:0/854235048 learned_addr learned my addr 172.21.15.167:0/854235048 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:24.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d4f427700 1 -- 172.21.15.167:0/854235048 >> v1:172.21.15.12:6789/0 conn(0x561d9446f400 legacy=0x561d9456e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d4f427700 1 -- 172.21.15.167:0/854235048 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d94481d40 con 0x561d9446f000 2024-09-13T23:43:24.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d4e425700 1 -- 172.21.15.167:0/854235048 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856652016 0 0) 0x561d945ca000 con 0x561d9446f400 2024-09-13T23:43:24.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d4f427700 1 --2- 172.21.15.167:0/854235048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f000 0x561d94754580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x561d9474fa70 tx=0x561d94573920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bbaa2c6254015fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d4e425700 1 -- 172.21.15.167:0/854235048 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d945134a0 con 0x561d9446f000 2024-09-13T23:43:24.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d4e425700 1 -- 172.21.15.167:0/854235048 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561d94533800 con 0x561d9446f000 2024-09-13T23:43:24.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d4e425700 1 -- 172.21.15.167:0/854235048 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d94513600 con 0x561d9446f000 2024-09-13T23:43:24.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/854235048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f000 msgr2=0x561d94754580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/854235048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f000 0x561d94754580 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x561d9474fa70 tx=0x561d94573920 comp rx=0 tx=0).stop 2024-09-13T23:43:24.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/854235048 shutdown_connections 2024-09-13T23:43:24.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/854235048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f000 0x561d94754580 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/854235048 >> 172.21.15.167:0/854235048 conn(0x561d9446ec00 msgr2=0x561d9456e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/854235048 shutdown_connections 2024-09-13T23:43:24.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.391+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/854235048 wait complete. 2024-09-13T23:43:24.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 Processor -- start 2024-09-13T23:43:24.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 -- start start 2024-09-13T23:43:24.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f000 0x561d94754580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d944fc500 con 0x561d9446f000 2024-09-13T23:43:24.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d4f427700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f000 0x561d94754580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d4f427700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f000 0x561d94754580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42896/0 (socket says 172.21.15.167:42896) 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d4f427700 1 -- 172.21.15.167:0/812915173 learned_addr learned my addr 172.21.15.167:0/812915173 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d4f427700 1 -- 172.21.15.167:0/812915173 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d94481ba0 con 0x561d9446f000 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d4f427700 1 --2- 172.21.15.167:0/812915173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f000 0x561d94754580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x561d95239e00 tx=0x561d94573800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb7822068a2f21d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d4dc24700 1 -- 172.21.15.167:0/812915173 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d94513760 con 0x561d9446f000 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d4dc24700 1 -- 172.21.15.167:0/812915173 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561d94533b00 con 0x561d9446f000 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d4dc24700 1 -- 172.21.15.167:0/812915173 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d945134a0 con 0x561d9446f000 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/812915173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f000 msgr2=0x561d94754580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/812915173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f000 0x561d94754580 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x561d95239e00 tx=0x561d94573800 comp rx=0 tx=0).stop 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/812915173 shutdown_connections 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/812915173 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f000 0x561d94754580 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/812915173 >> 172.21.15.167:0/812915173 conn(0x561d9446ec00 msgr2=0x561d9456f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/812915173 shutdown_connections 2024-09-13T23:43:24.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/812915173 wait complete. 2024-09-13T23:43:24.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 Processor -- start 2024-09-13T23:43:24.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 -- start start 2024-09-13T23:43:24.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f400 0x561d94754b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d50c6ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d944fc3c0 con 0x561d9446f400 2024-09-13T23:43:24.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d4f427700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f400 0x561d94754b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d4f427700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f400 0x561d94754b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42910/0 (socket says 172.21.15.167:42910) 2024-09-13T23:43:24.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.395+0000 7f5d4f427700 1 -- 172.21.15.167:0/3604045018 learned_addr learned my addr 172.21.15.167:0/3604045018 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d4f427700 1 -- 172.21.15.167:0/3604045018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d94481a00 con 0x561d9446f400 2024-09-13T23:43:24.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d4f427700 1 --2- 172.21.15.167:0/3604045018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f400 0x561d94754b00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x561d95239e90 tx=0x561d94573920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d4c421700 1 -- 172.21.15.167:0/3604045018 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d94513600 con 0x561d9446f400 2024-09-13T23:43:24.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3604045018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561d94481860 con 0x561d9446f400 2024-09-13T23:43:24.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d4c421700 1 -- 172.21.15.167:0/3604045018 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561d94533c80 con 0x561d9446f400 2024-09-13T23:43:24.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d4c421700 1 -- 172.21.15.167:0/3604045018 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d945138c0 con 0x561d9446f400 2024-09-13T23:43:24.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3604045018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d944816c0 con 0x561d9446f400 2024-09-13T23:43:24.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d50c6ef40 1 Processor -- start 2024-09-13T23:43:24.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d50c6ef40 1 -- start start 2024-09-13T23:43:24.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d50c6ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 0x561d94755080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d50c6ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d944fc000 con 0x561d9446fc00 2024-09-13T23:43:24.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d4c421700 1 -- 172.21.15.167:0/3604045018 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x561d94568b00 con 0x561d9446f400 2024-09-13T23:43:24.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d4c421700 1 --2- 172.21.15.167:0/3604045018 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d9480e000 0x561d94756100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d4c421700 1 -- 172.21.15.167:0/3604045018 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x561d944d5180 con 0x561d9446f400 2024-09-13T23:43:24.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d4fc28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 0x561d94755080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d4fc28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 0x561d94755080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42918/0 (socket says 172.21.15.167:42918) 2024-09-13T23:43:24.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d4fc28700 1 -- 172.21.15.167:0/3264354856 learned_addr learned my addr 172.21.15.167:0/3264354856 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.399+0000 7f5d4f427700 1 --2- 172.21.15.167:0/3604045018 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d9480e000 0x561d94756100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d4fc28700 1 -- 172.21.15.167:0/3264354856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d94480000 con 0x561d9446fc00 2024-09-13T23:43:24.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d4f427700 1 --2- 172.21.15.167:0/3604045018 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d9480e000 0x561d94756100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x561d94797650 tx=0x561d947d2b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d4fc28700 1 --2- 172.21.15.167:0/3264354856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 0x561d94755080 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x561d947962d0 tx=0x561d947d2e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f47f664e5e318b1f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d49c1c700 1 -- 172.21.15.167:0/3264354856 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d94513600 con 0x561d9446fc00 2024-09-13T23:43:24.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d49c1c700 1 -- 172.21.15.167:0/3264354856 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561d94822000 con 0x561d9446fc00 2024-09-13T23:43:24.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d49c1c700 1 -- 172.21.15.167:0/3264354856 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d945138c0 con 0x561d9446fc00 2024-09-13T23:43:24.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3264354856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 msgr2=0x561d94755080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/3264354856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 0x561d94755080 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x561d947962d0 tx=0x561d947d2e40 comp rx=0 tx=0).stop 2024-09-13T23:43:24.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3264354856 shutdown_connections 2024-09-13T23:43:24.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/3264354856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 0x561d94755080 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3264354856 >> 172.21.15.167:0/3264354856 conn(0x561d9446f800 msgr2=0x561d9456f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3264354856 shutdown_connections 2024-09-13T23:43:24.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3264354856 wait complete. 2024-09-13T23:43:24.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d50c6ef40 1 Processor -- start 2024-09-13T23:43:24.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d50c6ef40 1 -- start start 2024-09-13T23:43:24.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d50c6ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 0x561d94755080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d50c6ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d94494640 con 0x561d9446fc00 2024-09-13T23:43:24.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d4fc28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 0x561d94755080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d4fc28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 0x561d94755080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42924/0 (socket says 172.21.15.167:42924) 2024-09-13T23:43:24.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 learned_addr learned my addr 172.21.15.167:0/227947719 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.403+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d94480ea0 con 0x561d9446fc00 2024-09-13T23:43:24.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d4fc28700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 0x561d94755080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x561d94796870 tx=0x561d947d2a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d48419700 1 -- 172.21.15.167:0/227947719 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d94513760 con 0x561d9446fc00 2024-09-13T23:43:24.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d48419700 1 -- 172.21.15.167:0/227947719 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561d94822180 con 0x561d9446fc00 2024-09-13T23:43:24.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d48419700 1 -- 172.21.15.167:0/227947719 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d94513600 con 0x561d9446fc00 2024-09-13T23:43:24.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561d94480820 con 0x561d9446fc00 2024-09-13T23:43:24.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d944809c0 con 0x561d9446fc00 2024-09-13T23:43:24.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d48419700 1 -- 172.21.15.167:0/227947719 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x561d94569080 con 0x561d9446fc00 2024-09-13T23:43:24.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d48419700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d9480ec00 0x561d94756680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d48419700 1 -- 172.21.15.167:0/227947719 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x561d944d5340 con 0x561d9446fc00 2024-09-13T23:43:24.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d4ec26700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d9480ec00 0x561d94756680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d50c6ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561d9480f000 0x561d94755600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d9480f400 con 0x561d9480f000 2024-09-13T23:43:24.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d4fc28700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561d9480f000 0x561d94755600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.407+0000 7f5d4ec26700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d9480ec00 0x561d94756680 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x561d95238030 tx=0x561d948be0c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d4fc28700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561d9480f000 0x561d94755600 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 20 realm 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d9480f800 con 0x561d9480f000 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d9480fc00 con 0x561d9480f000 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561d9480f400 0x561d94755b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d9480f800 con 0x561d9480f400 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d4f427700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561d9480f400 0x561d94755b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d4f427700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561d9480f400 0x561d94755b80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:24.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.411+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d9480fc00 con 0x561d9480f400 2024-09-13T23:43:24.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d50c6ef40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:24.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d50c6ef40 20 searching for the correct realm 2024-09-13T23:43:24.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561d9480f800 0x561d94756c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d9480fc00 con 0x561d9480f800 2024-09-13T23:43:24.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4ec26700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561d9480f800 0x561d94756c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4ec26700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561d9480f800 0x561d94756c00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d9480fc00 con 0x561d9480f400 2024-09-13T23:43:24.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4f427700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561d9480fc00 0x561d94754580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480fc00 2024-09-13T23:43:24.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4fc28700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561d9480fc00 0x561d94754580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4fc28700 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561d9480fc00 0x561d94754580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f000 2024-09-13T23:43:24.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.415+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f000 2024-09-13T23:43:24.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.419+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.419+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f800 2024-09-13T23:43:24.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.419+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.419+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f000 2024-09-13T23:43:24.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.419+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.419+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f400 2024-09-13T23:43:24.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.419+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.419+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f400 2024-09-13T23:43:24.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.419+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.419+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f000 2024-09-13T23:43:24.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.419+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.419+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480fc00 2024-09-13T23:43:24.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.423+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.423+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f000 2024-09-13T23:43:24.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.423+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.423+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480fc00 2024-09-13T23:43:24.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.423+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.423+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f400 2024-09-13T23:43:24.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.423+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.423+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f800 2024-09-13T23:43:24.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.423+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.423+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f000 2024-09-13T23:43:24.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.423+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.423+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f400 2024-09-13T23:43:24.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.427+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.427+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f800 2024-09-13T23:43:24.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.427+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.427+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f000 2024-09-13T23:43:24.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.427+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.427+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f000 2024-09-13T23:43:24.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.427+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.427+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f000 2024-09-13T23:43:24.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.427+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.427+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f800 2024-09-13T23:43:24.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.431+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.431+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480fc00 2024-09-13T23:43:24.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.431+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.431+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f000 2024-09-13T23:43:24.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.431+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.431+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480fc00 2024-09-13T23:43:24.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.431+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.431+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f800 2024-09-13T23:43:24.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.431+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.431+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480fc00 2024-09-13T23:43:24.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.431+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.431+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f400 2024-09-13T23:43:24.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f400 2024-09-13T23:43:24.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f800 2024-09-13T23:43:24.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f400 2024-09-13T23:43:24.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f400 2024-09-13T23:43:24.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d50c6ef40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:24.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d50c6ef40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:24.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d50c6ef40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:24.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d50c6ef40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:24.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d50c6ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.435+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6800 con 0x561d9480f000 2024-09-13T23:43:24.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6c00 con 0x561d9480f400 2024-09-13T23:43:24.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:24.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7000 con 0x561d9480f400 2024-09-13T23:43:24.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:24.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 20 zone default found 2024-09-13T23:43:24.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 4 Realm: () 2024-09-13T23:43:24.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:24.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:24.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:24.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 20 zonegroup default 2024-09-13T23:43:24.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7400 con 0x561d9480fc00 2024-09-13T23:43:24.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7800 con 0x561d9480f800 2024-09-13T23:43:24.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.439+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f800 2024-09-13T23:43:24.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.443+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.443+0000 7f5d50c6ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.443+0000 7f5d50c6ef40 20 started sync module instance, tier type = 2024-09-13T23:43:24.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.443+0000 7f5d50c6ef40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:24.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.443+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7800 con 0x561d9480f400 2024-09-13T23:43:24.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.443+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.443+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94685044718336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f400 2024-09-13T23:43:24.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.443+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6400 con 0x561d9480f000 2024-09-13T23:43:24.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.447+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94685044718336] v33'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.447+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.447+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94685044719488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f000 2024-09-13T23:43:24.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.447+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6c00 con 0x561d9480f400 2024-09-13T23:43:24.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.451+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94685044719488] v33'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.451+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.451+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94685044720640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7000 con 0x561d9480f400 2024-09-13T23:43:24.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.451+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6800 con 0x561d9480fc00 2024-09-13T23:43:24.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.455+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94685044720640] v33'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.455+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.455+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94685044721792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7400 con 0x561d9480fc00 2024-09-13T23:43:24.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.455+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6400 con 0x561d9480f400 2024-09-13T23:43:24.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.455+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94685044721792] v33'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.455+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.459+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94685044722944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6000 con 0x561d9480f400 2024-09-13T23:43:24.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.459+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7800 con 0x561d9480f800 2024-09-13T23:43:24.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.459+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.459+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94685044724096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f800 2024-09-13T23:43:24.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.459+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6c00 con 0x561d9480f800 2024-09-13T23:43:24.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.459+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94685044722944] v33'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94685044724096] v33'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94685044725248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7000 con 0x561d9480f800 2024-09-13T23:43:24.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6400 con 0x561d9480fc00 2024-09-13T23:43:24.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94685044725248] v33'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'3 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94685044726400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7800 con 0x561d9480fc00 2024-09-13T23:43:24.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d50c6ef40 20 add_watcher() i=0 2024-09-13T23:43:24.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d50c6ef40 20 add_watcher() i=1 2024-09-13T23:43:24.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d50c6ef40 20 add_watcher() i=2 2024-09-13T23:43:24.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d50c6ef40 20 add_watcher() i=3 2024-09-13T23:43:24.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d50c6ef40 20 add_watcher() i=4 2024-09-13T23:43:24.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d50c6ef40 20 add_watcher() i=5 2024-09-13T23:43:24.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.463+0000 7f5d50c6ef40 20 add_watcher() i=6 2024-09-13T23:43:24.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.467+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94685044726400] v33'4 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.467+0000 7f5d50c6ef40 20 add_watcher() i=7 2024-09-13T23:43:24.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.467+0000 7f5d50c6ef40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:24.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.467+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f000 2024-09-13T23:43:24.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.467+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.467+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94685046390784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d6c00 con 0x561d9480f000 2024-09-13T23:43:24.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94685046390784] v33'24 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d363f5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:24.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d50c6ef40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:24.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d50c6ef40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:24.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d50c6ef40 1 Processor -- start 2024-09-13T23:43:24.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d50c6ef40 1 -- start start 2024-09-13T23:43:24.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d50c6ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 0x561d94757180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d50c6ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d94494780 con 0x561d948d7400 2024-09-13T23:43:24.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d4ec26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 0x561d94757180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d4ec26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 0x561d94757180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42934/0 (socket says 172.21.15.167:42934) 2024-09-13T23:43:24.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d4ec26700 1 -- 172.21.15.167:0/2415888975 learned_addr learned my addr 172.21.15.167:0/2415888975 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d4ec26700 1 -- 172.21.15.167:0/2415888975 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d94480b60 con 0x561d948d7400 2024-09-13T23:43:24.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d4ec26700 1 --2- 172.21.15.167:0/2415888975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 0x561d94757180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x561d94875f80 tx=0x561d948be060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852fea3604b8ff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d35bf4700 1 -- 172.21.15.167:0/2415888975 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d948f4000 con 0x561d948d7400 2024-09-13T23:43:24.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d35bf4700 1 -- 172.21.15.167:0/2415888975 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561d94822600 con 0x561d948d7400 2024-09-13T23:43:24.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.471+0000 7f5d35bf4700 1 -- 172.21.15.167:0/2415888975 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d948f4160 con 0x561d948d7400 2024-09-13T23:43:24.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2415888975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 msgr2=0x561d94757180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/2415888975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 0x561d94757180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x561d94875f80 tx=0x561d948be060 comp rx=0 tx=0).stop 2024-09-13T23:43:24.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2415888975 shutdown_connections 2024-09-13T23:43:24.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/2415888975 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 0x561d94757180 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2415888975 >> 172.21.15.167:0/2415888975 conn(0x561d948d6800 msgr2=0x561d94570000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2415888975 shutdown_connections 2024-09-13T23:43:24.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2415888975 wait complete. 2024-09-13T23:43:24.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 Processor -- start 2024-09-13T23:43:24.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 -- start start 2024-09-13T23:43:24.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 0x561d94757180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d944fc000 con 0x561d948d7400 2024-09-13T23:43:24.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d4ec26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 0x561d94757180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d4ec26700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 0x561d94757180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42942/0 (socket says 172.21.15.167:42942) 2024-09-13T23:43:24.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d4ec26700 1 -- 172.21.15.167:0/2119692298 learned_addr learned my addr 172.21.15.167:0/2119692298 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d4ec26700 1 -- 172.21.15.167:0/2119692298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d94480d00 con 0x561d948d7400 2024-09-13T23:43:24.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d4ec26700 1 --2- 172.21.15.167:0/2119692298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 0x561d94757180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x561d948ed260 tx=0x561d948be180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d343f1700 1 -- 172.21.15.167:0/2119692298 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d948f42c0 con 0x561d948d7400 2024-09-13T23:43:24.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d343f1700 1 -- 172.21.15.167:0/2119692298 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561d94822300 con 0x561d948d7400 2024-09-13T23:43:24.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2119692298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561d94480340 con 0x561d948d7400 2024-09-13T23:43:24.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d343f1700 1 -- 172.21.15.167:0/2119692298 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561d948f4000 con 0x561d948d7400 2024-09-13T23:43:24.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.475+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2119692298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d94480b60 con 0x561d948d7400 2024-09-13T23:43:24.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.479+0000 7f5d343f1700 1 -- 172.21.15.167:0/2119692298 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x561d94569b80 con 0x561d948d7400 2024-09-13T23:43:24.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.479+0000 7f5d343f1700 1 --2- 172.21.15.167:0/2119692298 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d94970000 0x561d94976000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.479+0000 7f5d343f1700 1 -- 172.21.15.167:0/2119692298 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x561d944d5500 con 0x561d948d7400 2024-09-13T23:43:24.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.479+0000 7f5d4fc28700 1 --2- 172.21.15.167:0/2119692298 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d94970000 0x561d94976000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.479+0000 7f5d50c6ef40 5 note: GC not initialized 2024-09-13T23:43:24.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.479+0000 7f5d4fc28700 1 --2- 172.21.15.167:0/2119692298 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d94970000 0x561d94976000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x561d94875b60 tx=0x561d947d3500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.479+0000 7f5d31bec700 20 reqs_thread_entry: start 2024-09-13T23:43:24.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.479+0000 7f5d50c6ef40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:24.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.479+0000 7f5d50c6ef40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:24.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.479+0000 7f5d2fbe8700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:54 2024 2024-09-13T23:43:24.490 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:24.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.479+0000 7f5d2f3c6700 20 reqs_thread_entry: start 2024-09-13T23:43:24.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.479+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f800 2024-09-13T23:43:24.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.483+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(62 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.483+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f400 2024-09-13T23:43:24.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.483+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(63 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.483+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f000 2024-09-13T23:43:24.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.483+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.483+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f000 2024-09-13T23:43:24.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.483+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(65 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.483+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480fc00 2024-09-13T23:43:24.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.483+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(66 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.483+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f400 2024-09-13T23:43:24.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.483+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.483+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f000 2024-09-13T23:43:24.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(68 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f800 2024-09-13T23:43:24.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(69 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480fc00 2024-09-13T23:43:24.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(70 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f400 2024-09-13T23:43:24.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(71 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f000 2024-09-13T23:43:24.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(72 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f800 2024-09-13T23:43:24.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(73 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f000 2024-09-13T23:43:24.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(74 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.487+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f800 2024-09-13T23:43:24.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(75 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480fc00 2024-09-13T23:43:24.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(76 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f800 2024-09-13T23:43:24.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(77 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f000 2024-09-13T23:43:24.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(78 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f400 2024-09-13T23:43:24.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(79 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480fc00 2024-09-13T23:43:24.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(80 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f400 2024-09-13T23:43:24.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(81 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.491+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f000 2024-09-13T23:43:24.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.495+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(82 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.495+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f800 2024-09-13T23:43:24.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.495+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(83 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.495+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480fc00 2024-09-13T23:43:24.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.495+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(84 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.495+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f000 2024-09-13T23:43:24.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.495+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(85 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.495+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f000 2024-09-13T23:43:24.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.495+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(86 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.495+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f800 2024-09-13T23:43:24.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.495+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(87 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.495+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f800 2024-09-13T23:43:24.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.499+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(88 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.499+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480fc00 2024-09-13T23:43:24.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.499+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(89 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.499+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f000 2024-09-13T23:43:24.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.499+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 28 ==== osd_op_reply(90 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.499+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f400 2024-09-13T23:43:24.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.499+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(91 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.499+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480f800 2024-09-13T23:43:24.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.499+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(92 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.499+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d948d7c00 con 0x561d9480fc00 2024-09-13T23:43:24.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.499+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 18 ==== osd_op_reply(93 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.503+0000 7f5d363f5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:24.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.503+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94685046390784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d94970400 con 0x561d9480f000 2024-09-13T23:43:24.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.503+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94685046390784] v33'25 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.503+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94685044718336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d94970800 con 0x561d9480f400 2024-09-13T23:43:24.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.507+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94685044718336] v33'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.507+0000 7f5d50c6ef40 20 remove_watcher() i=0 2024-09-13T23:43:24.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.507+0000 7f5d50c6ef40 2 removed watcher, disabling cache 2024-09-13T23:43:24.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.507+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94685044719488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d94970c00 con 0x561d9480f000 2024-09-13T23:43:24.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.511+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.1 v2:172.21.15.167:6800/3875661733 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94685044719488] v33'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480f000 2024-09-13T23:43:24.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.511+0000 7f5d50c6ef40 20 remove_watcher() i=1 2024-09-13T23:43:24.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.511+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94685044720640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d94971000 con 0x561d9480f400 2024-09-13T23:43:24.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.511+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94685044720640] v33'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.511+0000 7f5d50c6ef40 20 remove_watcher() i=2 2024-09-13T23:43:24.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.511+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94685044721792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d94971400 con 0x561d9480fc00 2024-09-13T23:43:24.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.515+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 19 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94685044721792] v33'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.515+0000 7f5d50c6ef40 20 remove_watcher() i=3 2024-09-13T23:43:24.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.515+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94685044722944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d94971800 con 0x561d9480f400 2024-09-13T23:43:24.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.519+0000 7f5d4f427700 1 -- 172.21.15.167:0/227947719 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94685044722944] v33'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3200 con 0x561d9480f400 2024-09-13T23:43:24.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.519+0000 7f5d50c6ef40 20 remove_watcher() i=4 2024-09-13T23:43:24.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.519+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94685044724096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d94971c00 con 0x561d9480f800 2024-09-13T23:43:24.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.519+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94685044724096] v33'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.519+0000 7f5d50c6ef40 20 remove_watcher() i=5 2024-09-13T23:43:24.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.519+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94685044725248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d94846000 con 0x561d9480f800 2024-09-13T23:43:24.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.523+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94685044725248] v33'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d3440 con 0x561d9480f800 2024-09-13T23:43:24.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.523+0000 7f5d50c6ef40 20 remove_watcher() i=6 2024-09-13T23:43:24.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.523+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94685044726400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561d94846400 con 0x561d9480fc00 2024-09-13T23:43:24.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.523+0000 7f5d4fc28700 1 -- 172.21.15.167:0/227947719 <== osd.2 v2:172.21.15.167:6808/1180369139 20 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94685044726400] v33'5 uv3 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561d948d2b40 con 0x561d9480fc00 2024-09-13T23:43:24.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.523+0000 7f5d50c6ef40 20 remove_watcher() i=7 2024-09-13T23:43:24.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.523+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561d9480f400 msgr2=0x561d94755b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.523+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561d9480f400 0x561d94755b80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561d9480f000 msgr2=0x561d94755600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561d9480f000 0x561d94755600 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561d9480fc00 msgr2=0x561d94754580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561d9480fc00 0x561d94754580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561d9480f800 msgr2=0x561d94756c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561d9480f800 0x561d94756c00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d9480ec00 msgr2=0x561d94756680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d9480ec00 0x561d94756680 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x561d95238030 tx=0x561d948be0c0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 msgr2=0x561d94755080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/227947719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446fc00 0x561d94755080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x561d94796870 tx=0x561d947d2a20 comp rx=0 tx=0).stop 2024-09-13T23:43:24.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 reap_dead start 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d4ec26700 1 -- 172.21.15.167:0/227947719 reap_dead start 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 shutdown_connections 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 >> 172.21.15.167:0/227947719 conn(0x561d9446f800 msgr2=0x561d9456f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 shutdown_connections 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/227947719 wait complete. 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2119692298 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d94970000 msgr2=0x561d94976000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/2119692298 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d94970000 0x561d94976000 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x561d94875b60 tx=0x561d947d3500 comp rx=0 tx=0).stop 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2119692298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 msgr2=0x561d94757180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/2119692298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 0x561d94757180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x561d948ed260 tx=0x561d948be180 comp rx=0 tx=0).stop 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2119692298 shutdown_connections 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/2119692298 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d94970000 0x561d94976000 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/2119692298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d948d7400 0x561d94757180 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2119692298 >> 172.21.15.167:0/2119692298 conn(0x561d948d6800 msgr2=0x561d94570000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2119692298 shutdown_connections 2024-09-13T23:43:24.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/2119692298 wait complete. 2024-09-13T23:43:24.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3604045018 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d9480e000 msgr2=0x561d94756100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/3604045018 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d9480e000 0x561d94756100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x561d94797650 tx=0x561d947d2b40 comp rx=0 tx=0).stop 2024-09-13T23:43:24.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3604045018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f400 msgr2=0x561d94754b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/3604045018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f400 0x561d94754b00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x561d95239e90 tx=0x561d94573920 comp rx=0 tx=0).stop 2024-09-13T23:43:24.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3604045018 shutdown_connections 2024-09-13T23:43:24.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/3604045018 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561d9480e000 0x561d94756100 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 --2- 172.21.15.167:0/3604045018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561d9446f400 0x561d94754b00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3604045018 >> 172.21.15.167:0/3604045018 conn(0x561d9446ec00 msgr2=0x561d9456f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3604045018 shutdown_connections 2024-09-13T23:43:24.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.527+0000 7f5d50c6ef40 1 -- 172.21.15.167:0/3604045018 wait complete. 2024-09-13T23:43:24.540 INFO:teuthology.orchestra.run.smithi167.stdout:[] 2024-09-13T23:43:24.543 DEBUG:tasks.util.rgw: json result: [] 2024-09-13T23:43:24.543 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud'] 2024-09-13T23:43:24.543 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud'] 2024-09-13T23:43:24.543 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud 2024-09-13T23:43:24.584 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:24.584 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:24.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.575+0000 7fc1d409df40 1 Processor -- start 2024-09-13T23:43:24.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.575+0000 7fc1d409df40 1 -- start start 2024-09-13T23:43:24.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.575+0000 7fc1d409df40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x562b62b43000 0x562b62e28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.575+0000 7fc1d409df40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562b62c07500 con 0x562b62b43400 2024-09-13T23:43:24.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.575+0000 7fc1d409df40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x562b62bd0a00 con 0x562b62b43000 2024-09-13T23:43:24.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.575+0000 7fc1d2856700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x562b62b43000 0x562b62e28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.575+0000 7fc1d2856700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x562b62b43000 0x562b62e28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42948/0 (socket says 172.21.15.167:42948) 2024-09-13T23:43:24.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.575+0000 7fc1d2856700 1 -- 172.21.15.167:0/2071433633 learned_addr learned my addr 172.21.15.167:0/2071433633 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.575+0000 7fc1d2055700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x562b62b43400 0x562b62c42800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:40290/0 (socket says 172.21.15.167:40290) 2024-09-13T23:43:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d2856700 1 -- 172.21.15.167:0/2071433633 >> v1:172.21.15.12:6789/0 conn(0x562b62b43400 legacy=0x562b62c42800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d2856700 1 -- 172.21.15.167:0/2071433633 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b62b55d40 con 0x562b62b43000 2024-09-13T23:43:24.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d1854700 1 -- 172.21.15.167:0/2071433633 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483065920 0 0) 0x562b62c9e000 con 0x562b62b43400 2024-09-13T23:43:24.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d2856700 1 --2- 172.21.15.167:0/2071433633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x562b62e23a70 tx=0x562b62c47920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388bf38976147f8c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d1854700 1 -- 172.21.15.167:0/2071433633 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62be74a0 con 0x562b62b43000 2024-09-13T23:43:24.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d1854700 1 -- 172.21.15.167:0/2071433633 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b62c07800 con 0x562b62b43000 2024-09-13T23:43:24.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d1854700 1 -- 172.21.15.167:0/2071433633 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62be7600 con 0x562b62b43000 2024-09-13T23:43:24.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d409df40 1 -- 172.21.15.167:0/2071433633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 msgr2=0x562b62e28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2071433633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x562b62e23a70 tx=0x562b62c47920 comp rx=0 tx=0).stop 2024-09-13T23:43:24.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d409df40 1 -- 172.21.15.167:0/2071433633 shutdown_connections 2024-09-13T23:43:24.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2071433633 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d409df40 1 -- 172.21.15.167:0/2071433633 >> 172.21.15.167:0/2071433633 conn(0x562b62b42c00 msgr2=0x562b62c42000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d409df40 1 -- 172.21.15.167:0/2071433633 shutdown_connections 2024-09-13T23:43:24.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d409df40 1 -- 172.21.15.167:0/2071433633 wait complete. 2024-09-13T23:43:24.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d409df40 1 Processor -- start 2024-09-13T23:43:24.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d409df40 1 -- start start 2024-09-13T23:43:24.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d409df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d409df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62bd0500 con 0x562b62b43000 2024-09-13T23:43:24.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d2856700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d2856700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42962/0 (socket says 172.21.15.167:42962) 2024-09-13T23:43:24.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d2856700 1 -- 172.21.15.167:0/241677799 learned_addr learned my addr 172.21.15.167:0/241677799 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d2856700 1 -- 172.21.15.167:0/241677799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b62b55ba0 con 0x562b62b43000 2024-09-13T23:43:24.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.579+0000 7fc1d2856700 1 --2- 172.21.15.167:0/241677799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x562b62e23f50 tx=0x562b62c47800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb3385375a3608e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d1053700 1 -- 172.21.15.167:0/241677799 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62be7760 con 0x562b62b43000 2024-09-13T23:43:24.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d1053700 1 -- 172.21.15.167:0/241677799 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b62c07b00 con 0x562b62b43000 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d1053700 1 -- 172.21.15.167:0/241677799 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62be74a0 con 0x562b62b43000 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 -- 172.21.15.167:0/241677799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 msgr2=0x562b62e28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 --2- 172.21.15.167:0/241677799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x562b62e23f50 tx=0x562b62c47800 comp rx=0 tx=0).stop 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 -- 172.21.15.167:0/241677799 shutdown_connections 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 --2- 172.21.15.167:0/241677799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 -- 172.21.15.167:0/241677799 >> 172.21.15.167:0/241677799 conn(0x562b62b42c00 msgr2=0x562b62c43000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 -- 172.21.15.167:0/241677799 shutdown_connections 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 -- 172.21.15.167:0/241677799 wait complete. 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 Processor -- start 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 -- start start 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62bd03c0 con 0x562b62b43000 2024-09-13T23:43:24.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d2856700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d2856700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42976/0 (socket says 172.21.15.167:42976) 2024-09-13T23:43:24.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d2856700 1 -- 172.21.15.167:0/1341593727 learned_addr learned my addr 172.21.15.167:0/1341593727 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d2856700 1 -- 172.21.15.167:0/1341593727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b62b55a00 con 0x562b62b43000 2024-09-13T23:43:24.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d2856700 1 --2- 172.21.15.167:0/1341593727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x562b63909f50 tx=0x562b62c47920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1cf850700 1 -- 172.21.15.167:0/1341593727 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62be7600 con 0x562b62b43000 2024-09-13T23:43:24.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1cf850700 1 -- 172.21.15.167:0/1341593727 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b62c07c80 con 0x562b62b43000 2024-09-13T23:43:24.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 -- 172.21.15.167:0/1341593727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562b62b55860 con 0x562b62b43000 2024-09-13T23:43:24.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1cf850700 1 -- 172.21.15.167:0/1341593727 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62be78c0 con 0x562b62b43000 2024-09-13T23:43:24.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 -- 172.21.15.167:0/1341593727 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b62b556c0 con 0x562b62b43000 2024-09-13T23:43:24.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 Processor -- start 2024-09-13T23:43:24.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.583+0000 7fc1d409df40 1 -- start start 2024-09-13T23:43:24.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 0x562b62e28b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62b68640 con 0x562b62b43800 2024-09-13T23:43:24.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d3057700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 0x562b62e28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d3057700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 0x562b62e28b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42978/0 (socket says 172.21.15.167:42978) 2024-09-13T23:43:24.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d3057700 1 -- 172.21.15.167:0/1724099372 learned_addr learned my addr 172.21.15.167:0/1724099372 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1cf850700 1 -- 172.21.15.167:0/1341593727 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x562b62c3cb00 con 0x562b62b43000 2024-09-13T23:43:24.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1cf850700 1 --2- 172.21.15.167:0/1341593727 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b62eec000 0x562b62e29080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1cf850700 1 -- 172.21.15.167:0/1341593727 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x562b62ba9180 con 0x562b62b43000 2024-09-13T23:43:24.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d3057700 1 -- 172.21.15.167:0/1724099372 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b62b54000 con 0x562b62b43800 2024-09-13T23:43:24.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d2856700 1 --2- 172.21.15.167:0/1341593727 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b62eec000 0x562b62e29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d3057700 1 --2- 172.21.15.167:0/1724099372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 0x562b62e28b00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x562b62e6d620 tx=0x562b62c47ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd031411b62cd8de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1cd04b700 1 -- 172.21.15.167:0/1724099372 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62be7600 con 0x562b62b43800 2024-09-13T23:43:24.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1cd04b700 1 -- 172.21.15.167:0/1724099372 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b62efe000 con 0x562b62b43800 2024-09-13T23:43:24.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1cd04b700 1 -- 172.21.15.167:0/1724099372 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62be78c0 con 0x562b62b43800 2024-09-13T23:43:24.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 -- 172.21.15.167:0/1724099372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 msgr2=0x562b62e28b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 --2- 172.21.15.167:0/1724099372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 0x562b62e28b00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x562b62e6d620 tx=0x562b62c47ce0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d2856700 1 --2- 172.21.15.167:0/1341593727 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b62eec000 0x562b62e29080 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x562b62e6d560 tx=0x562b62eb0c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 -- 172.21.15.167:0/1724099372 shutdown_connections 2024-09-13T23:43:24.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 --2- 172.21.15.167:0/1724099372 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 0x562b62e28b00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 -- 172.21.15.167:0/1724099372 >> 172.21.15.167:0/1724099372 conn(0x562b62b43400 msgr2=0x562b62c43800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 -- 172.21.15.167:0/1724099372 shutdown_connections 2024-09-13T23:43:24.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 -- 172.21.15.167:0/1724099372 wait complete. 2024-09-13T23:43:24.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 Processor -- start 2024-09-13T23:43:24.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 -- start start 2024-09-13T23:43:24.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 0x562b62e28b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d409df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62b688c0 con 0x562b62b43800 2024-09-13T23:43:24.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d3057700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 0x562b62e28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d3057700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 0x562b62e28b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42986/0 (socket says 172.21.15.167:42986) 2024-09-13T23:43:24.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.587+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 learned_addr learned my addr 172.21.15.167:0/353595876 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b62b54ea0 con 0x562b62b43800 2024-09-13T23:43:24.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1d3057700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 0x562b62e28b00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x562b63909e60 tx=0x562b62c47da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1cb848700 1 -- 172.21.15.167:0/353595876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62be7760 con 0x562b62b43800 2024-09-13T23:43:24.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1cb848700 1 -- 172.21.15.167:0/353595876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b62efe180 con 0x562b62b43800 2024-09-13T23:43:24.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1cb848700 1 -- 172.21.15.167:0/353595876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62be7600 con 0x562b62b43800 2024-09-13T23:43:24.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562b62b54820 con 0x562b62b43800 2024-09-13T23:43:24.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b62b549c0 con 0x562b62b43800 2024-09-13T23:43:24.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1cb848700 1 -- 172.21.15.167:0/353595876 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x562b62c3d080 con 0x562b62b43800 2024-09-13T23:43:24.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1cb848700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b62eecc00 0x562b62e29600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1cb848700 1 -- 172.21.15.167:0/353595876 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x562b62ba9340 con 0x562b62b43800 2024-09-13T23:43:24.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1d2055700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b62eecc00 0x562b62e29600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1d2055700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b62eecc00 0x562b62e29600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x562b62ecdb30 tx=0x562b62f8c060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1d409df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1d409df40 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562b62eed000 0x562b62e29b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.591+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed400 con 0x562b62eed000 2024-09-13T23:43:24.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d3057700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562b62eed000 0x562b62e29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d3057700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562b62eed000 0x562b62e29b80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d409df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d409df40 20 realm 2024-09-13T23:43:24.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d409df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed000 2024-09-13T23:43:24.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d409df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d409df40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:24.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d409df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eedc00 con 0x562b62eed000 2024-09-13T23:43:24.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d409df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d409df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d409df40 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562b62eed400 0x562b62e2a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed400 2024-09-13T23:43:24.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d2856700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562b62eed400 0x562b62e2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.595+0000 7fc1d2856700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562b62eed400 0x562b62e2a100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d409df40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:24.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d409df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eedc00 con 0x562b62eed400 2024-09-13T23:43:24.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d409df40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:24.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d409df40 20 searching for the correct realm 2024-09-13T23:43:24.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d409df40 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562b62faa000 0x562b62e2a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62faa400 con 0x562b62faa000 2024-09-13T23:43:24.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d2055700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562b62faa000 0x562b62e2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d2055700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562b62faa000 0x562b62e2a680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62faa400 con 0x562b62eed400 2024-09-13T23:43:24.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d2856700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562b62faa400 0x562b62e2ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa400 2024-09-13T23:43:24.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.599+0000 7fc1d3057700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562b62faa400 0x562b62e2ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d3057700 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562b62faa400 0x562b62e2ac00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed000 2024-09-13T23:43:24.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed000 2024-09-13T23:43:24.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa000 2024-09-13T23:43:24.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed000 2024-09-13T23:43:24.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed400 2024-09-13T23:43:24.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.603+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed400 2024-09-13T23:43:24.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed000 2024-09-13T23:43:24.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa400 2024-09-13T23:43:24.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed000 2024-09-13T23:43:24.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa400 2024-09-13T23:43:24.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed400 2024-09-13T23:43:24.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa000 2024-09-13T23:43:24.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.607+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed000 2024-09-13T23:43:24.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.611+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.611+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed400 2024-09-13T23:43:24.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.611+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.611+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa000 2024-09-13T23:43:24.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.611+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.611+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed000 2024-09-13T23:43:24.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.611+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.611+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed000 2024-09-13T23:43:24.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.611+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.611+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed000 2024-09-13T23:43:24.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.611+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.611+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa000 2024-09-13T23:43:24.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.615+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.615+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa400 2024-09-13T23:43:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.615+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.615+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed000 2024-09-13T23:43:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.615+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.615+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa400 2024-09-13T23:43:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.615+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.615+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa000 2024-09-13T23:43:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.615+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.615+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa400 2024-09-13T23:43:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.615+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.615+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed400 2024-09-13T23:43:24.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed400 2024-09-13T23:43:24.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa000 2024-09-13T23:43:24.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed400 2024-09-13T23:43:24.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62eed400 2024-09-13T23:43:24.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d409df40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:24.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d409df40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:24.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d409df40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:24.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d409df40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:24.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d409df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62faa800 con 0x562b62eed000 2024-09-13T23:43:24.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d409df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d409df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.619+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62faac00 con 0x562b62eed400 2024-09-13T23:43:24.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:24.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab000 con 0x562b62eed400 2024-09-13T23:43:24.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:24.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 20 zone default found 2024-09-13T23:43:24.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 4 Realm: () 2024-09-13T23:43:24.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:24.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:24.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:24.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 20 zonegroup default 2024-09-13T23:43:24.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab400 con 0x562b62faa400 2024-09-13T23:43:24.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab800 con 0x562b62faa000 2024-09-13T23:43:24.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.623+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fabc00 con 0x562b62faa000 2024-09-13T23:43:24.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.627+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.627+0000 7fc1d409df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.627+0000 7fc1d409df40 20 started sync module instance, tier type = 2024-09-13T23:43:24.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.627+0000 7fc1d409df40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:24.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.627+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab800 con 0x562b62eed400 2024-09-13T23:43:24.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.627+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.627+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94744342559488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fabc00 con 0x562b62eed400 2024-09-13T23:43:24.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.627+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62faac00 con 0x562b62eed000 2024-09-13T23:43:24.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.631+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94744342559488] v33'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.631+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.631+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94744342560640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab000 con 0x562b62eed000 2024-09-13T23:43:24.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.631+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab800 con 0x562b62eed400 2024-09-13T23:43:24.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.635+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94744342560640] v33'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.635+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.635+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94744342561792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eedc00 con 0x562b62eed400 2024-09-13T23:43:24.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.635+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eed800 con 0x562b62faa400 2024-09-13T23:43:24.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.635+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.635+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94744342562944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62faa800 con 0x562b62faa400 2024-09-13T23:43:24.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.639+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab400 con 0x562b62eed400 2024-09-13T23:43:24.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.639+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94744342561792] v33'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.639+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94744342562944] v33'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.639+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.639+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94744342564096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62faac00 con 0x562b62eed400 2024-09-13T23:43:24.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.639+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab000 con 0x562b62faa000 2024-09-13T23:43:24.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.643+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.643+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94744342565248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb0000 con 0x562b62faa000 2024-09-13T23:43:24.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.643+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb0400 con 0x562b62faa000 2024-09-13T23:43:24.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.643+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94744342564096] v33'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.643+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94744342565248] v33'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.643+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.643+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94744342566400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb0800 con 0x562b62faa000 2024-09-13T23:43:24.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.643+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb0c00 con 0x562b62faa400 2024-09-13T23:43:24.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'6 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94744342567552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb1000 con 0x562b62faa400 2024-09-13T23:43:24.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d409df40 20 add_watcher() i=0 2024-09-13T23:43:24.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d409df40 20 add_watcher() i=1 2024-09-13T23:43:24.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d409df40 20 add_watcher() i=2 2024-09-13T23:43:24.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d409df40 20 add_watcher() i=3 2024-09-13T23:43:24.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d409df40 20 add_watcher() i=4 2024-09-13T23:43:24.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d409df40 20 add_watcher() i=5 2024-09-13T23:43:24.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94744342566400] v33'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d409df40 20 add_watcher() i=6 2024-09-13T23:43:24.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94744342567552] v33'7 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d409df40 20 add_watcher() i=7 2024-09-13T23:43:24.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d409df40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:24.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb1400 con 0x562b62eed000 2024-09-13T23:43:24.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.647+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94744344223744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb1800 con 0x562b62eed000 2024-09-13T23:43:24.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.651+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94744344223744] v33'26 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.651+0000 7fc1b9824700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:24.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.651+0000 7fc1d409df40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:24.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.651+0000 7fc1d409df40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:24.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.651+0000 7fc1d409df40 1 Processor -- start 2024-09-13T23:43:24.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.651+0000 7fc1d409df40 1 -- start start 2024-09-13T23:43:24.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.651+0000 7fc1d409df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 0x562b62e2b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.651+0000 7fc1d409df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62bd03c0 con 0x562b62fb0c00 2024-09-13T23:43:24.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.651+0000 7fc1d2055700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 0x562b62e2b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.651+0000 7fc1d2055700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 0x562b62e2b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42988/0 (socket says 172.21.15.167:42988) 2024-09-13T23:43:24.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.651+0000 7fc1d2055700 1 -- 172.21.15.167:0/501094931 learned_addr learned my addr 172.21.15.167:0/501094931 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d2055700 1 -- 172.21.15.167:0/501094931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b62b54b60 con 0x562b62fb0c00 2024-09-13T23:43:24.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d2055700 1 --2- 172.21.15.167:0/501094931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 0x562b62e2b180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x562b62fbe090 tx=0x562b62f8c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bf8317df744598 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1b9023700 1 -- 172.21.15.167:0/501094931 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62fc6000 con 0x562b62fb0c00 2024-09-13T23:43:24.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1b9023700 1 -- 172.21.15.167:0/501094931 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b62efe600 con 0x562b62fb0c00 2024-09-13T23:43:24.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1b9023700 1 -- 172.21.15.167:0/501094931 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62fc6160 con 0x562b62fb0c00 2024-09-13T23:43:24.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d409df40 1 -- 172.21.15.167:0/501094931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 msgr2=0x562b62e2b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d409df40 1 --2- 172.21.15.167:0/501094931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 0x562b62e2b180 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x562b62fbe090 tx=0x562b62f8c1e0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d409df40 1 -- 172.21.15.167:0/501094931 shutdown_connections 2024-09-13T23:43:24.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d409df40 1 --2- 172.21.15.167:0/501094931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 0x562b62e2b180 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d409df40 1 -- 172.21.15.167:0/501094931 >> 172.21.15.167:0/501094931 conn(0x562b62fb1c00 msgr2=0x562b62c44000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d409df40 1 -- 172.21.15.167:0/501094931 shutdown_connections 2024-09-13T23:43:24.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d409df40 1 -- 172.21.15.167:0/501094931 wait complete. 2024-09-13T23:43:24.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d409df40 1 Processor -- start 2024-09-13T23:43:24.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d409df40 1 -- start start 2024-09-13T23:43:24.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d409df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 0x562b62e2b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d409df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b62bd0500 con 0x562b62fb0c00 2024-09-13T23:43:24.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d2055700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 0x562b62e2b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d2055700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 0x562b62e2b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42994/0 (socket says 172.21.15.167:42994) 2024-09-13T23:43:24.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d2055700 1 -- 172.21.15.167:0/2674633440 learned_addr learned my addr 172.21.15.167:0/2674633440 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.655+0000 7fc1d2055700 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b62b54d00 con 0x562b62fb0c00 2024-09-13T23:43:24.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1d2055700 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 0x562b62e2b180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x562b62fbf350 tx=0x562b62f8c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1b7820700 1 -- 172.21.15.167:0/2674633440 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62fc62c0 con 0x562b62fb0c00 2024-09-13T23:43:24.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562b62b54340 con 0x562b62fb0c00 2024-09-13T23:43:24.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1b7820700 1 -- 172.21.15.167:0/2674633440 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b62efe300 con 0x562b62fb0c00 2024-09-13T23:43:24.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1b7820700 1 -- 172.21.15.167:0/2674633440 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562b62fc6000 con 0x562b62fb0c00 2024-09-13T23:43:24.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b62b54b60 con 0x562b62fb0c00 2024-09-13T23:43:24.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1b7820700 1 -- 172.21.15.167:0/2674633440 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x562b62c3db80 con 0x562b62fb0c00 2024-09-13T23:43:24.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1b7820700 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b63042000 0x562b63048000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1b7820700 1 -- 172.21.15.167:0/2674633440 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x562b62ba9500 con 0x562b62fb0c00 2024-09-13T23:43:24.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1d3057700 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b63042000 0x562b63048000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562b62fb1000 0x562b62e2b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb1400 con 0x562b62fb1000 2024-09-13T23:43:24.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1d2856700 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562b62fb1000 0x562b62e2b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1d3057700 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b63042000 0x562b63048000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x562b62e6cde0 tx=0x562b62eb14a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.659+0000 7fc1d2856700 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562b62fb1000 0x562b62e2b700 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.663+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.663+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562b62fb0000 0x562b63048580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.663+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb0400 con 0x562b62fb0000 2024-09-13T23:43:24.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.663+0000 7fc1d2055700 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562b62fb0000 0x562b63048580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.663+0000 7fc1d2055700 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562b62fb0000 0x562b63048580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.667+0000 7fc1d2055700 1 -- 172.21.15.167:0/2674633440 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'10 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562b62fa1440 con 0x562b62fb0000 2024-09-13T23:43:24.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.667+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562b62faa800 0x562b63048b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.667+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab000 con 0x562b62faa800 2024-09-13T23:43:24.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.667+0000 7fc1d3057700 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562b62faa800 0x562b63048b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.667+0000 7fc1d3057700 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562b62faa800 0x562b63048b00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.667+0000 7fc1d3057700 1 -- 172.21.15.167:0/2674633440 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'10 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa800 2024-09-13T23:43:24.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.667+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562b62eed800 0x562b63049080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.667+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63042400 con 0x562b62eed800 2024-09-13T23:43:24.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.667+0000 7fc1d2856700 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562b62eed800 0x562b63049080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.671+0000 7fc1d2856700 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562b62eed800 0x562b63049080 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.671+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'27 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed800 2024-09-13T23:43:24.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.671+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63042800 con 0x562b62fb1000 2024-09-13T23:43:24.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.675+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'20 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.675+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63042c00 con 0x562b62faa800 2024-09-13T23:43:24.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.675+0000 7fc1d3057700 1 -- 172.21.15.167:0/2674633440 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'13 uv10 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa800 2024-09-13T23:43:24.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.675+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63043000 con 0x562b62fb1000 2024-09-13T23:43:24.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.679+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'12 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.679+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63043400 con 0x562b62eed800 2024-09-13T23:43:24.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.683+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'13 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed800 2024-09-13T23:43:24.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.683+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63043800 con 0x562b62fb1000 2024-09-13T23:43:24.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.687+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'10 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.687+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63043c00 con 0x562b62fb1000 2024-09-13T23:43:24.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.687+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'16 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.687+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb1400 con 0x562b62fb1000 2024-09-13T23:43:24.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.691+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'21 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.691+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fabc00 con 0x562b62faa800 2024-09-13T23:43:24.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.691+0000 7fc1d3057700 1 -- 172.21.15.167:0/2674633440 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'12 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa800 2024-09-13T23:43:24.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.691+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab800 con 0x562b62fb1000 2024-09-13T23:43:24.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.695+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'10 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.695+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab400 con 0x562b62eed800 2024-09-13T23:43:24.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.699+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'19 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed800 2024-09-13T23:43:24.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.699+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62faac00 con 0x562b62eed800 2024-09-13T23:43:24.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.699+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'28 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed800 2024-09-13T23:43:24.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.699+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eedc00 con 0x562b62fb1000 2024-09-13T23:43:24.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.703+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'14 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.703+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb1400 con 0x562b62fb1000 2024-09-13T23:43:24.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.707+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'22 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.707+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63042400 con 0x562b62faa800 2024-09-13T23:43:24.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.707+0000 7fc1d3057700 1 -- 172.21.15.167:0/2674633440 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'10 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa800 2024-09-13T23:43:24.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.707+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63042800 con 0x562b62fb1000 2024-09-13T23:43:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.711+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.711+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63043000 con 0x562b62eed800 2024-09-13T23:43:24.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.715+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'29 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed800 2024-09-13T23:43:24.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.715+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63043400 con 0x562b62fb1000 2024-09-13T23:43:24.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.719+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'15 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.719+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63043800 con 0x562b62fb1000 2024-09-13T23:43:24.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.719+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'20 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.719+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63043c00 con 0x562b62faa800 2024-09-13T23:43:24.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.723+0000 7fc1d3057700 1 -- 172.21.15.167:0/2674633440 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa800 2024-09-13T23:43:24.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.723+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63042c00 con 0x562b62fb1000 2024-09-13T23:43:24.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.727+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.727+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63042400 con 0x562b62faa800 2024-09-13T23:43:24.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.727+0000 7fc1d3057700 1 -- 172.21.15.167:0/2674633440 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'6 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa800 2024-09-13T23:43:24.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.727+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63043c00 con 0x562b62faa800 2024-09-13T23:43:24.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.731+0000 7fc1d3057700 1 -- 172.21.15.167:0/2674633440 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'14 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa800 2024-09-13T23:43:24.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.731+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63042800 con 0x562b62faa800 2024-09-13T23:43:24.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.735+0000 7fc1d3057700 1 -- 172.21.15.167:0/2674633440 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa800 2024-09-13T23:43:24.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.735+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63043000 con 0x562b62fb1000 2024-09-13T23:43:24.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.735+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'10 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.735+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b63043400 con 0x562b62eed800 2024-09-13T23:43:24.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.739+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'14 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed800 2024-09-13T23:43:24.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.739+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb1800 con 0x562b62fb1000 2024-09-13T23:43:24.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.743+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'16 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62fb1000 2024-09-13T23:43:24.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.743+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fb1400 con 0x562b62eed800 2024-09-13T23:43:24.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.743+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'20 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed800 2024-09-13T23:43:24.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.743+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62eedc00 con 0x562b62fb0000 2024-09-13T23:43:24.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.747+0000 7fc1d2055700 1 -- 172.21.15.167:0/2674633440 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562b62fa1440 con 0x562b62fb0000 2024-09-13T23:43:24.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1b501b700 20 reqs_thread_entry: start 2024-09-13T23:43:24.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1d409df40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:24.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1d409df40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:24.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1b3017700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:55 2024 2024-09-13T23:43:24.759 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:24.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1b27f5700 20 reqs_thread_entry: start 2024-09-13T23:43:24.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1d409df40 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-09-13T23:43:24.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1d409df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab800 con 0x562b62faa400 2024-09-13T23:43:24.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 bud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1d409df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1d409df40 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x0 2024-09-13T23:43:24.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1d409df40 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-09-13T23:43:24.761 INFO:teuthology.orchestra.run.smithi167.stderr:could not remove user: unable to remove user, user does not exist 2024-09-13T23:43:24.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1b9824700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:24.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.751+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94744344223744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62faac00 con 0x562b62eed000 2024-09-13T23:43:24.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.755+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94744344223744] v33'30 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.755+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94744342559488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62fab400 con 0x562b62eed400 2024-09-13T23:43:24.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.759+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94744342559488] v33'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.759+0000 7fc1d409df40 20 remove_watcher() i=0 2024-09-13T23:43:24.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.759+0000 7fc1d409df40 2 removed watcher, disabling cache 2024-09-13T23:43:24.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.759+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94744342560640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62f4e000 con 0x562b62eed000 2024-09-13T23:43:24.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.759+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94744342560640] v33'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62eed000 2024-09-13T23:43:24.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.759+0000 7fc1d409df40 20 remove_watcher() i=1 2024-09-13T23:43:24.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.759+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94744342561792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62f4e400 con 0x562b62eed400 2024-09-13T23:43:24.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.763+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94744342561792] v33'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.763+0000 7fc1d409df40 20 remove_watcher() i=2 2024-09-13T23:43:24.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.763+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94744342562944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62f4e800 con 0x562b62faa400 2024-09-13T23:43:24.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.767+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94744342562944] v33'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.767+0000 7fc1d409df40 20 remove_watcher() i=3 2024-09-13T23:43:24.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.767+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94744342564096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62f4ec00 con 0x562b62eed400 2024-09-13T23:43:24.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.767+0000 7fc1d2856700 1 -- 172.21.15.167:0/353595876 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94744342564096] v33'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1200 con 0x562b62eed400 2024-09-13T23:43:24.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.767+0000 7fc1d409df40 20 remove_watcher() i=4 2024-09-13T23:43:24.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.767+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94744342565248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62f4f000 con 0x562b62faa000 2024-09-13T23:43:24.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.771+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94744342565248] v33'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.771+0000 7fc1d409df40 20 remove_watcher() i=5 2024-09-13T23:43:24.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.771+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94744342566400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62f4f400 con 0x562b62faa000 2024-09-13T23:43:24.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.771+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94744342566400] v33'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa1440 con 0x562b62faa000 2024-09-13T23:43:24.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.771+0000 7fc1d409df40 20 remove_watcher() i=6 2024-09-13T23:43:24.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.771+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94744342567552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562b62f4f800 con 0x562b62faa400 2024-09-13T23:43:24.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d3057700 1 -- 172.21.15.167:0/353595876 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94744342567552] v33'8 uv6 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562b62fa0b40 con 0x562b62faa400 2024-09-13T23:43:24.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 20 remove_watcher() i=7 2024-09-13T23:43:24.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562b62eed400 msgr2=0x562b62e2a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562b62eed400 0x562b62e2a100 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562b62eed000 msgr2=0x562b62e29b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562b62eed000 0x562b62e29b80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562b62faa400 msgr2=0x562b62e2ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562b62faa400 0x562b62e2ac00 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562b62faa000 msgr2=0x562b62e2a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562b62faa000 0x562b62e2a680 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b62eecc00 msgr2=0x562b62e29600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b62eecc00 0x562b62e29600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x562b62ecdb30 tx=0x562b62f8c060 comp rx=0 tx=0).stop 2024-09-13T23:43:24.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 msgr2=0x562b62e28b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 0x562b62e28b00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x562b63909e60 tx=0x562b62c47da0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d2055700 1 -- 172.21.15.167:0/353595876 reap_dead start 2024-09-13T23:43:24.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 shutdown_connections 2024-09-13T23:43:24.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 --2- 172.21.15.167:0/353595876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43800 0x562b62e28b00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 >> 172.21.15.167:0/353595876 conn(0x562b62b43400 msgr2=0x562b62c43800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 shutdown_connections 2024-09-13T23:43:24.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 -- 172.21.15.167:0/353595876 wait complete. 2024-09-13T23:43:24.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562b62fb1000 msgr2=0x562b62e2b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562b62fb1000 0x562b62e2b700 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562b62eed800 msgr2=0x562b63049080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.775+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562b62eed800 0x562b63049080 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562b62fb0000 msgr2=0x562b63048580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562b62fb0000 0x562b63048580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562b62faa800 msgr2=0x562b63048b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562b62faa800 0x562b63048b00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b63042000 msgr2=0x562b63048000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b63042000 0x562b63048000 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x562b62e6cde0 tx=0x562b62eb14a0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 msgr2=0x562b62e2b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d2856700 1 -- 172.21.15.167:0/2674633440 reap_dead start 2024-09-13T23:43:24.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 0x562b62e2b180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x562b62fbf350 tx=0x562b62f8c0c0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 shutdown_connections 2024-09-13T23:43:24.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 --2- 172.21.15.167:0/2674633440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62fb0c00 0x562b62e2b180 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 >> 172.21.15.167:0/2674633440 conn(0x562b62fb1c00 msgr2=0x562b62c44000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 shutdown_connections 2024-09-13T23:43:24.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/2674633440 wait complete. 2024-09-13T23:43:24.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/1341593727 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b62eec000 msgr2=0x562b62e29080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 --2- 172.21.15.167:0/1341593727 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b62eec000 0x562b62e29080 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x562b62e6d560 tx=0x562b62eb0c60 comp rx=0 tx=0).stop 2024-09-13T23:43:24.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/1341593727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 msgr2=0x562b62e28580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 --2- 172.21.15.167:0/1341593727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x562b63909f50 tx=0x562b62c47920 comp rx=0 tx=0).stop 2024-09-13T23:43:24.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/1341593727 shutdown_connections 2024-09-13T23:43:24.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 --2- 172.21.15.167:0/1341593727 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562b62eec000 0x562b62e29080 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 --2- 172.21.15.167:0/1341593727 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562b62b43000 0x562b62e28580 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/1341593727 >> 172.21.15.167:0/1341593727 conn(0x562b62b42c00 msgr2=0x562b62c43000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/1341593727 shutdown_connections 2024-09-13T23:43:24.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.779+0000 7fc1d409df40 1 -- 172.21.15.167:0/1341593727 wait complete. 2024-09-13T23:43:24.793 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-13T23:43:24.794 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bar'] 2024-09-13T23:43:24.794 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bar'] 2024-09-13T23:43:24.794 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bar 2024-09-13T23:43:24.876 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:24.876 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:24.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.867+0000 7f015b4bef40 1 Processor -- start 2024-09-13T23:43:24.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.867+0000 7f015b4bef40 1 -- start start 2024-09-13T23:43:24.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.867+0000 7f015b4bef40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56027799b000 0x560277c80580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.867+0000 7f015b4bef40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x560277a5f500 con 0x56027799b400 2024-09-13T23:43:24.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.867+0000 7f015b4bef40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x560277a28a00 con 0x56027799b000 2024-09-13T23:43:24.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.867+0000 7f0159c77700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56027799b000 0x560277c80580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.867+0000 7f0159c77700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56027799b000 0x560277c80580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43004/0 (socket says 172.21.15.167:43004) 2024-09-13T23:43:24.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.867+0000 7f0159476700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x56027799b400 0x560277a9a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:40300/0 (socket says 172.21.15.167:40300) 2024-09-13T23:43:24.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.867+0000 7f0159c77700 1 -- 172.21.15.167:0/2854980782 learned_addr learned my addr 172.21.15.167:0/2854980782 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f0159c77700 1 -- 172.21.15.167:0/2854980782 >> v1:172.21.15.12:6789/0 conn(0x56027799b400 legacy=0x560277a9a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f0159c77700 1 -- 172.21.15.167:0/2854980782 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602779add40 con 0x56027799b000 2024-09-13T23:43:24.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f0159c77700 1 --2- 172.21.15.167:0/2854980782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b000 0x560277c80580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x560277c7ba70 tx=0x560277a9f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61a7c439142342d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f0158c75700 1 -- 172.21.15.167:0/2854980782 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277a3f4a0 con 0x56027799b000 2024-09-13T23:43:24.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f0158c75700 1 -- 172.21.15.167:0/2854980782 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560277a5f800 con 0x56027799b000 2024-09-13T23:43:24.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f0158c75700 1 -- 172.21.15.167:0/2854980782 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277a3f600 con 0x56027799b000 2024-09-13T23:43:24.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f015b4bef40 1 -- 172.21.15.167:0/2854980782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b000 msgr2=0x560277c80580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f015b4bef40 1 --2- 172.21.15.167:0/2854980782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b000 0x560277c80580 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x560277c7ba70 tx=0x560277a9f920 comp rx=0 tx=0).stop 2024-09-13T23:43:24.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f015b4bef40 1 -- 172.21.15.167:0/2854980782 shutdown_connections 2024-09-13T23:43:24.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f015b4bef40 1 --2- 172.21.15.167:0/2854980782 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b000 0x560277c80580 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f015b4bef40 1 -- 172.21.15.167:0/2854980782 >> 172.21.15.167:0/2854980782 conn(0x56027799ac00 msgr2=0x560277a9a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f015b4bef40 1 -- 172.21.15.167:0/2854980782 shutdown_connections 2024-09-13T23:43:24.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f015b4bef40 1 -- 172.21.15.167:0/2854980782 wait complete. 2024-09-13T23:43:24.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f015b4bef40 1 Processor -- start 2024-09-13T23:43:24.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f015b4bef40 1 -- start start 2024-09-13T23:43:24.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f015b4bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b400 0x560277c80b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f015b4bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560277a28500 con 0x56027799b400 2024-09-13T23:43:24.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f0159c77700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b400 0x560277c80b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f0159c77700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b400 0x560277c80b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43016/0 (socket says 172.21.15.167:43016) 2024-09-13T23:43:24.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f0159c77700 1 -- 172.21.15.167:0/664985679 learned_addr learned my addr 172.21.15.167:0/664985679 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.871+0000 7f0159c77700 1 -- 172.21.15.167:0/664985679 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602779adba0 con 0x56027799b400 2024-09-13T23:43:24.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f0159c77700 1 --2- 172.21.15.167:0/664985679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b400 0x560277c80b00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x560277af7cb0 tx=0x560277a9f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9f60bb6095ab80 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f0158474700 1 -- 172.21.15.167:0/664985679 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277a3f760 con 0x56027799b400 2024-09-13T23:43:24.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f0158474700 1 -- 172.21.15.167:0/664985679 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560277a5fb00 con 0x56027799b400 2024-09-13T23:43:24.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f0158474700 1 -- 172.21.15.167:0/664985679 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277a3f4a0 con 0x56027799b400 2024-09-13T23:43:24.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f015b4bef40 1 -- 172.21.15.167:0/664985679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b400 msgr2=0x560277c80b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f015b4bef40 1 --2- 172.21.15.167:0/664985679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b400 0x560277c80b00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x560277af7cb0 tx=0x560277a9f800 comp rx=0 tx=0).stop 2024-09-13T23:43:24.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f015b4bef40 1 -- 172.21.15.167:0/664985679 shutdown_connections 2024-09-13T23:43:24.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f015b4bef40 1 --2- 172.21.15.167:0/664985679 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b400 0x560277c80b00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f015b4bef40 1 -- 172.21.15.167:0/664985679 >> 172.21.15.167:0/664985679 conn(0x56027799ac00 msgr2=0x560277a9b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f015b4bef40 1 -- 172.21.15.167:0/664985679 shutdown_connections 2024-09-13T23:43:24.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f015b4bef40 1 -- 172.21.15.167:0/664985679 wait complete. 2024-09-13T23:43:24.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f015b4bef40 1 Processor -- start 2024-09-13T23:43:24.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f015b4bef40 1 -- start start 2024-09-13T23:43:24.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f015b4bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b800 0x560277c80580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f015b4bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560277a283c0 con 0x56027799b800 2024-09-13T23:43:24.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f0159c77700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b800 0x560277c80580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f0159c77700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b800 0x560277c80580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43028/0 (socket says 172.21.15.167:43028) 2024-09-13T23:43:24.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f0159c77700 1 -- 172.21.15.167:0/1556211171 learned_addr learned my addr 172.21.15.167:0/1556211171 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f0159c77700 1 -- 172.21.15.167:0/1556211171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602779ada00 con 0x56027799b800 2024-09-13T23:43:24.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f0159c77700 1 --2- 172.21.15.167:0/1556211171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b800 0x560277c80580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x560277af7e30 tx=0x560277a9f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.875+0000 7f0156c71700 1 -- 172.21.15.167:0/1556211171 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277a3f600 con 0x56027799b800 2024-09-13T23:43:24.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015b4bef40 1 -- 172.21.15.167:0/1556211171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5602779ad860 con 0x56027799b800 2024-09-13T23:43:24.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f0156c71700 1 -- 172.21.15.167:0/1556211171 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560277a5fc80 con 0x56027799b800 2024-09-13T23:43:24.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f0156c71700 1 -- 172.21.15.167:0/1556211171 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277a3f8c0 con 0x56027799b800 2024-09-13T23:43:24.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015b4bef40 1 -- 172.21.15.167:0/1556211171 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602779ad6c0 con 0x56027799b800 2024-09-13T23:43:24.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015b4bef40 1 Processor -- start 2024-09-13T23:43:24.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015b4bef40 1 -- start start 2024-09-13T23:43:24.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015b4bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 0x560277c80b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015b4bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602779c0640 con 0x560277ce6000 2024-09-13T23:43:24.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f0156c71700 1 -- 172.21.15.167:0/1556211171 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x560277a94b00 con 0x56027799b800 2024-09-13T23:43:24.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015a478700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 0x560277c80b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015a478700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 0x560277c80b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43030/0 (socket says 172.21.15.167:43030) 2024-09-13T23:43:24.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015a478700 1 -- 172.21.15.167:0/3770453665 learned_addr learned my addr 172.21.15.167:0/3770453665 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f0156c71700 1 --2- 172.21.15.167:0/1556211171 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277ce7400 0x560277c81600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f0156c71700 1 -- 172.21.15.167:0/1556211171 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x560277a01180 con 0x56027799b800 2024-09-13T23:43:24.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015a478700 1 -- 172.21.15.167:0/3770453665 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602779ac000 con 0x560277ce6000 2024-09-13T23:43:24.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f0159c77700 1 --2- 172.21.15.167:0/1556211171 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277ce7400 0x560277c81600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015a478700 1 --2- 172.21.15.167:0/3770453665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 0x560277c80b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x560277d22060 tx=0x560277a9fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce308c6df1e845c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f0159c77700 1 --2- 172.21.15.167:0/1556211171 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277ce7400 0x560277c81600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x560277cbf3b0 tx=0x560277d04c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015446c700 1 -- 172.21.15.167:0/3770453665 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277a3f600 con 0x560277ce6000 2024-09-13T23:43:24.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015446c700 1 -- 172.21.15.167:0/3770453665 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560277d5a000 con 0x560277ce6000 2024-09-13T23:43:24.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.879+0000 7f015446c700 1 -- 172.21.15.167:0/3770453665 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277a3f8c0 con 0x560277ce6000 2024-09-13T23:43:24.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 -- 172.21.15.167:0/3770453665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 msgr2=0x560277c80b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3770453665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 0x560277c80b00 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x560277d22060 tx=0x560277a9fce0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 -- 172.21.15.167:0/3770453665 shutdown_connections 2024-09-13T23:43:24.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3770453665 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 0x560277c80b00 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 -- 172.21.15.167:0/3770453665 >> 172.21.15.167:0/3770453665 conn(0x56027799bc00 msgr2=0x560277a9b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 -- 172.21.15.167:0/3770453665 shutdown_connections 2024-09-13T23:43:24.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 -- 172.21.15.167:0/3770453665 wait complete. 2024-09-13T23:43:24.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 Processor -- start 2024-09-13T23:43:24.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 -- start start 2024-09-13T23:43:24.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 0x560277c80b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602779c08c0 con 0x560277ce6000 2024-09-13T23:43:24.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015a478700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 0x560277c80b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015a478700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 0x560277c80b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43042/0 (socket says 172.21.15.167:43042) 2024-09-13T23:43:24.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 learned_addr learned my addr 172.21.15.167:0/684889021 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602779acea0 con 0x560277ce6000 2024-09-13T23:43:24.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015a478700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 0x560277c80b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x560277cbee70 tx=0x560277a9fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f0152c69700 1 -- 172.21.15.167:0/684889021 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277a3f760 con 0x560277ce6000 2024-09-13T23:43:24.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f0152c69700 1 -- 172.21.15.167:0/684889021 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560277d5a180 con 0x560277ce6000 2024-09-13T23:43:24.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5602779ac820 con 0x560277ce6000 2024-09-13T23:43:24.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f0152c69700 1 -- 172.21.15.167:0/684889021 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277a3f600 con 0x560277ce6000 2024-09-13T23:43:24.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.883+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602779ac9c0 con 0x560277ce6000 2024-09-13T23:43:24.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f0152c69700 1 -- 172.21.15.167:0/684889021 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x560277a94dc0 con 0x560277ce6000 2024-09-13T23:43:24.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f0152c69700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277de2000 0x560277c81b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f0152c69700 1 -- 172.21.15.167:0/684889021 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x560277a01340 con 0x560277ce6000 2024-09-13T23:43:24.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560277ce6400 0x560277c81080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f0159476700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277de2000 0x560277c81b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ce6800 con 0x560277ce6400 2024-09-13T23:43:24.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015a478700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560277ce6400 0x560277c81080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f0159476700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277de2000 0x560277c81b80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x560277c7bf80 tx=0x560277df40c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015a478700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560277ce6400 0x560277c81080 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 20 realm 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ce6c00 con 0x560277ce6400 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ce7000 con 0x560277ce6400 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.887+0000 7f015b4bef40 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560277de2400 0x560277c82100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2800 con 0x560277de2400 2024-09-13T23:43:24.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f0159c77700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560277de2400 0x560277c82100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f0159c77700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560277de2400 0x560277c82100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f015b4bef40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:24.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f015b4bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2400 2024-09-13T23:43:24.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f015b4bef40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:24.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f015b4bef40 20 searching for the correct realm 2024-09-13T23:43:24.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f015b4bef40 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560277ce6800 0x560277c82680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ce6c00 con 0x560277ce6800 2024-09-13T23:43:24.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f0159476700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560277ce6800 0x560277c82680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.891+0000 7f0159476700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560277ce6800 0x560277c82680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ce6c00 con 0x560277de2400 2024-09-13T23:43:24.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f0159c77700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560277de2800 0x560277c83700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2800 2024-09-13T23:43:24.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f015a478700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560277de2800 0x560277c83700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f015a478700 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560277de2800 0x560277c83700 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:24.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6400 2024-09-13T23:43:24.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6400 2024-09-13T23:43:24.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6800 2024-09-13T23:43:24.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.895+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6400 2024-09-13T23:43:24.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.899+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.899+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2400 2024-09-13T23:43:24.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.899+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.899+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2400 2024-09-13T23:43:24.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.899+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.899+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6400 2024-09-13T23:43:24.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.899+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.899+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2800 2024-09-13T23:43:24.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.899+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:24.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.899+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6400 2024-09-13T23:43:24.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.899+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.899+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2800 2024-09-13T23:43:24.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:24.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2400 2024-09-13T23:43:24.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6800 2024-09-13T23:43:24.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6400 2024-09-13T23:43:24.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2400 2024-09-13T23:43:24.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6800 2024-09-13T23:43:24.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6400 2024-09-13T23:43:24.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.903+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6400 2024-09-13T23:43:24.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6400 2024-09-13T23:43:24.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6800 2024-09-13T23:43:24.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2800 2024-09-13T23:43:24.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:24.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6400 2024-09-13T23:43:24.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2800 2024-09-13T23:43:24.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:24.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6800 2024-09-13T23:43:24.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.907+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2800 2024-09-13T23:43:24.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.911+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:24.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.911+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2400 2024-09-13T23:43:24.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.911+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.911+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2400 2024-09-13T23:43:24.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.911+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.911+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277ce6800 2024-09-13T23:43:24.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.911+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.911+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2400 2024-09-13T23:43:24.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.911+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.911+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2400 2024-09-13T23:43:24.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.911+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:24.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:24.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:24.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:24.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ce7000 con 0x560277ce6400 2024-09-13T23:43:24.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3000 con 0x560277de2400 2024-09-13T23:43:24.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:24.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3400 con 0x560277de2400 2024-09-13T23:43:24.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:24.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 zone default found 2024-09-13T23:43:24.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 4 Realm: () 2024-09-13T23:43:24.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:24.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:24.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:24.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 zonegroup default 2024-09-13T23:43:24.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3800 con 0x560277de2800 2024-09-13T23:43:24.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:24.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.915+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3c00 con 0x560277ce6800 2024-09-13T23:43:24.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.919+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.919+0000 7f015b4bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.919+0000 7f015b4bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:24.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.919+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e10000 con 0x560277ce6800 2024-09-13T23:43:24.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.919+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.919+0000 7f015b4bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:24.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.919+0000 7f015b4bef40 20 started sync module instance, tier type = 2024-09-13T23:43:24.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.919+0000 7f015b4bef40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:24.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.919+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e10400 con 0x560277de2400 2024-09-13T23:43:24.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.919+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.919+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94568599485184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e10800 con 0x560277de2400 2024-09-13T23:43:24.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.919+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e10c00 con 0x560277ce6400 2024-09-13T23:43:24.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.923+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.923+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94568599486336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e11000 con 0x560277ce6400 2024-09-13T23:43:24.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.923+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e11400 con 0x560277de2400 2024-09-13T23:43:24.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.923+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94568599485184] v33'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.927+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94568599486336] v33'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.927+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.927+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94568599487488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e11800 con 0x560277de2400 2024-09-13T23:43:24.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.927+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e11c00 con 0x560277de2800 2024-09-13T23:43:24.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.927+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:24.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.927+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94568599488640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e10c00 con 0x560277de2800 2024-09-13T23:43:24.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.927+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3800 con 0x560277de2400 2024-09-13T23:43:24.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.931+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94568599487488] v33'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.931+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94568599488640] v33'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:24.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.931+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.931+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94568599489792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de2400 2024-09-13T23:43:24.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.931+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3000 con 0x560277ce6800 2024-09-13T23:43:24.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.935+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94568599489792] v33'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:24.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.935+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.935+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94568599490944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3400 con 0x560277ce6800 2024-09-13T23:43:24.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.935+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3c00 con 0x560277ce6800 2024-09-13T23:43:24.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.935+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.935+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94568599492096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ce7000 con 0x560277ce6800 2024-09-13T23:43:24.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.935+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ce6c00 con 0x560277de2800 2024-09-13T23:43:24.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.935+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94568599490944] v33'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'9 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:24.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94568599493248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56027799b400 con 0x560277de2800 2024-09-13T23:43:24.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015b4bef40 20 add_watcher() i=0 2024-09-13T23:43:24.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015b4bef40 20 add_watcher() i=1 2024-09-13T23:43:24.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015b4bef40 20 add_watcher() i=2 2024-09-13T23:43:24.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015b4bef40 20 add_watcher() i=3 2024-09-13T23:43:24.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015b4bef40 20 add_watcher() i=4 2024-09-13T23:43:24.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015b4bef40 20 add_watcher() i=5 2024-09-13T23:43:24.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94568599492096] v33'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:24.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015b4bef40 20 add_watcher() i=6 2024-09-13T23:43:24.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94568599493248] v33'10 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:24.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015b4bef40 20 add_watcher() i=7 2024-09-13T23:43:24.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015b4bef40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:24.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e11000 con 0x560277ce6400 2024-09-13T23:43:24.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.939+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94568601182208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e10400 con 0x560277ce6400 2024-09-13T23:43:24.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94568601182208] v33'31 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:24.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f0140c45700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:24.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f015b4bef40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:24.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f015b4bef40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:24.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f015b4bef40 1 Processor -- start 2024-09-13T23:43:24.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f015b4bef40 1 -- start start 2024-09-13T23:43:24.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f015b4bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 0x560277c82c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f015b4bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560277a283c0 con 0x560277de3000 2024-09-13T23:43:24.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f0159476700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 0x560277c82c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f0159476700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 0x560277c82c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43048/0 (socket says 172.21.15.167:43048) 2024-09-13T23:43:24.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f0159476700 1 -- 172.21.15.167:0/3918068150 learned_addr learned my addr 172.21.15.167:0/3918068150 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f0159476700 1 -- 172.21.15.167:0/3918068150 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602779acb60 con 0x560277de3000 2024-09-13T23:43:24.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f0159476700 1 --2- 172.21.15.167:0/3918068150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 0x560277c82c00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x560277e20060 tx=0x560277df4060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672861f0fbcc54b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f0140444700 1 -- 172.21.15.167:0/3918068150 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277e28000 con 0x560277de3000 2024-09-13T23:43:24.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f0140444700 1 -- 172.21.15.167:0/3918068150 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560277d5a600 con 0x560277de3000 2024-09-13T23:43:24.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.943+0000 7f0140444700 1 -- 172.21.15.167:0/3918068150 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277e28160 con 0x560277de3000 2024-09-13T23:43:24.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 -- 172.21.15.167:0/3918068150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 msgr2=0x560277c82c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:24.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3918068150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 0x560277c82c00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x560277e20060 tx=0x560277df4060 comp rx=0 tx=0).stop 2024-09-13T23:43:24.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 -- 172.21.15.167:0/3918068150 shutdown_connections 2024-09-13T23:43:24.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3918068150 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 0x560277c82c00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:24.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 -- 172.21.15.167:0/3918068150 >> 172.21.15.167:0/3918068150 conn(0x560277e10000 msgr2=0x560277a9c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:24.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 -- 172.21.15.167:0/3918068150 shutdown_connections 2024-09-13T23:43:24.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 -- 172.21.15.167:0/3918068150 wait complete. 2024-09-13T23:43:24.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 Processor -- start 2024-09-13T23:43:24.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 -- start start 2024-09-13T23:43:24.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 0x560277c82c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560277a28500 con 0x560277de3000 2024-09-13T23:43:24.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f0159476700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 0x560277c82c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f0159476700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 0x560277c82c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43056/0 (socket says 172.21.15.167:43056) 2024-09-13T23:43:24.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f0159476700 1 -- 172.21.15.167:0/3379302867 learned_addr learned my addr 172.21.15.167:0/3379302867 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:24.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f0159476700 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602779acd00 con 0x560277de3000 2024-09-13T23:43:24.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f0159476700 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 0x560277c82c00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x560277e21320 tx=0x560277df4180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f013ec41700 1 -- 172.21.15.167:0/3379302867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277e282c0 con 0x560277de3000 2024-09-13T23:43:24.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f013ec41700 1 -- 172.21.15.167:0/3379302867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560277d5a300 con 0x560277de3000 2024-09-13T23:43:24.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5602779ac340 con 0x560277de3000 2024-09-13T23:43:24.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f013ec41700 1 -- 172.21.15.167:0/3379302867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560277e28000 con 0x560277de3000 2024-09-13T23:43:24.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.947+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602779acb60 con 0x560277de3000 2024-09-13T23:43:24.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.951+0000 7f013ec41700 1 -- 172.21.15.167:0/3379302867 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x560277a95b80 con 0x560277de3000 2024-09-13T23:43:24.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.951+0000 7f013ec41700 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277ce6c00 0x560277ea6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.951+0000 7f013ec41700 1 -- 172.21.15.167:0/3379302867 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x560277a01500 con 0x560277de3000 2024-09-13T23:43:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.951+0000 7f015a478700 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277ce6c00 0x560277ea6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.951+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560277de3400 0x560277c83180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.951+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3c00 con 0x560277de3400 2024-09-13T23:43:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.951+0000 7f0159c77700 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560277de3400 0x560277c83180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.951+0000 7f015a478700 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277ce6c00 0x560277ea6000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x560277d81bf0 tx=0x560277d05500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.951+0000 7f0159c77700 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560277de3400 0x560277c83180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.951+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:24.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.951+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560277e11c00 0x560277ea6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.951+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e10c00 con 0x560277e11c00 2024-09-13T23:43:24.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.955+0000 7f0159476700 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560277e11c00 0x560277ea6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.955+0000 7f0159476700 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560277e11c00 0x560277ea6580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.955+0000 7f0159476700 1 -- 172.21.15.167:0/3379302867 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'11 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560277da5440 con 0x560277e11c00 2024-09-13T23:43:24.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.955+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560277e11000 0x560277ea6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.955+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56027799b400 con 0x560277e11000 2024-09-13T23:43:24.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.955+0000 7f015a478700 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560277e11000 0x560277ea6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.955+0000 7f015a478700 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560277e11000 0x560277ea6b00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.959+0000 7f015a478700 1 -- 172.21.15.167:0/3379302867 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'11 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277e11000 2024-09-13T23:43:24.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.959+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560277eca000 0x560277ea7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:24.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.959+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277eca400 con 0x560277eca000 2024-09-13T23:43:24.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.959+0000 7f0159c77700 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560277eca000 0x560277ea7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:24.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.959+0000 7f0159c77700 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560277eca000 0x560277ea7080 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:24.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.963+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'32 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277eca000 2024-09-13T23:43:24.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.963+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277eca800 con 0x560277de3400 2024-09-13T23:43:24.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.963+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'23 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:24.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.963+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecac00 con 0x560277e11000 2024-09-13T23:43:24.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.967+0000 7f015a478700 1 -- 172.21.15.167:0/3379302867 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'15 uv10 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277e11000 2024-09-13T23:43:24.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.967+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecb000 con 0x560277de3400 2024-09-13T23:43:24.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.967+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'13 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:24.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.967+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecb400 con 0x560277eca000 2024-09-13T23:43:24.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.971+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'15 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277eca000 2024-09-13T23:43:24.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.971+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecb800 con 0x560277de3400 2024-09-13T23:43:24.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.975+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:24.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.975+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecbc00 con 0x560277de3400 2024-09-13T23:43:24.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.975+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'17 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:24.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.975+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277eca400 con 0x560277de3400 2024-09-13T23:43:24.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.979+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'24 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:24.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.979+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3800 con 0x560277e11000 2024-09-13T23:43:24.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.983+0000 7f015a478700 1 -- 172.21.15.167:0/3379302867 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'13 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277e11000 2024-09-13T23:43:24.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.983+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277de3400 2024-09-13T23:43:24.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.983+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:24.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.983+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3c00 con 0x560277eca000 2024-09-13T23:43:24.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.987+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'21 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277eca000 2024-09-13T23:43:24.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.987+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e10800 con 0x560277eca000 2024-09-13T23:43:24.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.991+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'33 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277eca000 2024-09-13T23:43:24.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.991+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e11400 con 0x560277de3400 2024-09-13T23:43:25.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.991+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:25.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.991+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e11800 con 0x560277de3400 2024-09-13T23:43:25.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.995+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'25 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:25.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.995+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277e11000 2024-09-13T23:43:25.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.999+0000 7f015a478700 1 -- 172.21.15.167:0/3379302867 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'11 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277e11000 2024-09-13T23:43:25.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.999+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3c00 con 0x560277de3400 2024-09-13T23:43:25.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.999+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:25.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:24.999+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277eca800 con 0x560277eca000 2024-09-13T23:43:25.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.003+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'34 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277eca000 2024-09-13T23:43:25.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.003+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecb000 con 0x560277de3400 2024-09-13T23:43:25.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.007+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'18 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:25.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.007+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecb400 con 0x560277de3400 2024-09-13T23:43:25.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.007+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'21 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:25.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.007+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecb800 con 0x560277e11000 2024-09-13T23:43:25.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.011+0000 7f015a478700 1 -- 172.21.15.167:0/3379302867 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277e11000 2024-09-13T23:43:25.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.011+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecbc00 con 0x560277de3400 2024-09-13T23:43:25.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.011+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:25.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.015+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277eca400 con 0x560277e11000 2024-09-13T23:43:25.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.015+0000 7f015a478700 1 -- 172.21.15.167:0/3379302867 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'7 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277e11000 2024-09-13T23:43:25.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.015+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecac00 con 0x560277e11000 2024-09-13T23:43:25.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.019+0000 7f015a478700 1 -- 172.21.15.167:0/3379302867 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'16 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277e11000 2024-09-13T23:43:25.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.019+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecb800 con 0x560277e11000 2024-09-13T23:43:25.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.019+0000 7f015a478700 1 -- 172.21.15.167:0/3379302867 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277e11000 2024-09-13T23:43:25.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.019+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3800 con 0x560277de3400 2024-09-13T23:43:25.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.023+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'11 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:25.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.023+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de2c00 con 0x560277eca000 2024-09-13T23:43:25.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.027+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'16 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277eca000 2024-09-13T23:43:25.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.027+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56027799b400 con 0x560277de3400 2024-09-13T23:43:25.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.027+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'19 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de3400 2024-09-13T23:43:25.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.027+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e10400 con 0x560277eca000 2024-09-13T23:43:25.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.031+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'22 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277eca000 2024-09-13T23:43:25.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.031+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277de3c00 con 0x560277e11c00 2024-09-13T23:43:25.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.035+0000 7f0159476700 1 -- 172.21.15.167:0/3379302867 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560277da5440 con 0x560277e11c00 2024-09-13T23:43:25.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f013c43c700 20 reqs_thread_entry: start 2024-09-13T23:43:25.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f015b4bef40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:25.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f015b4bef40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:25.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f013a438700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:55 2024 2024-09-13T23:43:25.048 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:25.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f0139c16700 20 reqs_thread_entry: start 2024-09-13T23:43:25.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f015b4bef40 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-09-13T23:43:25.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f015b4bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecb000 con 0x560277de2800 2024-09-13T23:43:25.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:25.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f015b4bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f015b4bef40 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-09-13T23:43:25.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f015b4bef40 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-09-13T23:43:25.049 INFO:teuthology.orchestra.run.smithi167.stderr:could not remove user: unable to remove user, user does not exist 2024-09-13T23:43:25.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f0140c45700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:25.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.039+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94568601182208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecb400 con 0x560277ce6400 2024-09-13T23:43:25.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.043+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94568601182208] v33'35 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:25.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.043+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94568599485184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecbc00 con 0x560277de2400 2024-09-13T23:43:25.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.047+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94568599485184] v33'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:25.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.047+0000 7f015b4bef40 20 remove_watcher() i=0 2024-09-13T23:43:25.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.047+0000 7f015b4bef40 2 removed watcher, disabling cache 2024-09-13T23:43:25.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.047+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94568599486336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e10800 con 0x560277ce6400 2024-09-13T23:43:25.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.047+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94568599486336] v33'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277ce6400 2024-09-13T23:43:25.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.047+0000 7f015b4bef40 20 remove_watcher() i=1 2024-09-13T23:43:25.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.047+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94568599487488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e11400 con 0x560277de2400 2024-09-13T23:43:25.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.051+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94568599487488] v33'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:25.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.051+0000 7f015b4bef40 20 remove_watcher() i=2 2024-09-13T23:43:25.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.051+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94568599488640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e11800 con 0x560277de2800 2024-09-13T23:43:25.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.051+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94568599488640] v33'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:25.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.051+0000 7f015b4bef40 20 remove_watcher() i=3 2024-09-13T23:43:25.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.051+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94568599489792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e10800 con 0x560277de2400 2024-09-13T23:43:25.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.055+0000 7f0159c77700 1 -- 172.21.15.167:0/684889021 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94568599489792] v33'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5200 con 0x560277de2400 2024-09-13T23:43:25.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.055+0000 7f015b4bef40 20 remove_watcher() i=4 2024-09-13T23:43:25.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.055+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94568599490944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277e11800 con 0x560277ce6800 2024-09-13T23:43:25.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.059+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94568599490944] v33'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:25.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.059+0000 7f015b4bef40 20 remove_watcher() i=5 2024-09-13T23:43:25.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.059+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94568599492096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277eca400 con 0x560277ce6800 2024-09-13T23:43:25.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.059+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94568599492096] v33'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da5440 con 0x560277ce6800 2024-09-13T23:43:25.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.059+0000 7f015b4bef40 20 remove_watcher() i=6 2024-09-13T23:43:25.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.059+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94568599493248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560277ecac00 con 0x560277de2800 2024-09-13T23:43:25.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015a478700 1 -- 172.21.15.167:0/684889021 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94568599493248] v33'11 uv9 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560277da4b40 con 0x560277de2800 2024-09-13T23:43:25.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 20 remove_watcher() i=7 2024-09-13T23:43:25.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560277de2400 msgr2=0x560277c82100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560277de2400 0x560277c82100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560277ce6400 msgr2=0x560277c81080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560277ce6400 0x560277c81080 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560277de2800 msgr2=0x560277c83700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560277de2800 0x560277c83700 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560277ce6800 msgr2=0x560277c82680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560277ce6800 0x560277c82680 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277de2000 msgr2=0x560277c81b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277de2000 0x560277c81b80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x560277c7bf80 tx=0x560277df40c0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 msgr2=0x560277c80b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/684889021 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277ce6000 0x560277c80b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x560277cbee70 tx=0x560277a9fc20 comp rx=0 tx=0).stop 2024-09-13T23:43:25.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 reap_dead start 2024-09-13T23:43:25.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f0159476700 1 -- 172.21.15.167:0/684889021 reap_dead start 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 shutdown_connections 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 >> 172.21.15.167:0/684889021 conn(0x56027799bc00 msgr2=0x560277a9b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 shutdown_connections 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/684889021 wait complete. 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560277de3400 msgr2=0x560277c83180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560277de3400 0x560277c83180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560277eca000 msgr2=0x560277ea7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560277eca000 0x560277ea7080 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560277e11c00 msgr2=0x560277ea6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560277e11c00 0x560277ea6580 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560277e11000 msgr2=0x560277ea6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560277e11000 0x560277ea6b00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277ce6c00 msgr2=0x560277ea6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277ce6c00 0x560277ea6000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x560277d81bf0 tx=0x560277d05500 comp rx=0 tx=0).stop 2024-09-13T23:43:25.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 msgr2=0x560277c82c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 0x560277c82c00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x560277e21320 tx=0x560277df4180 comp rx=0 tx=0).stop 2024-09-13T23:43:25.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f0159c77700 1 -- 172.21.15.167:0/3379302867 reap_dead start 2024-09-13T23:43:25.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 shutdown_connections 2024-09-13T23:43:25.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 --2- 172.21.15.167:0/3379302867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560277de3000 0x560277c82c00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 >> 172.21.15.167:0/3379302867 conn(0x560277e10000 msgr2=0x560277a9c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 shutdown_connections 2024-09-13T23:43:25.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.063+0000 7f015b4bef40 1 -- 172.21.15.167:0/3379302867 wait complete. 2024-09-13T23:43:25.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.067+0000 7f015b4bef40 1 -- 172.21.15.167:0/1556211171 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277ce7400 msgr2=0x560277c81600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.067+0000 7f015b4bef40 1 --2- 172.21.15.167:0/1556211171 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277ce7400 0x560277c81600 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x560277cbf3b0 tx=0x560277d04c60 comp rx=0 tx=0).stop 2024-09-13T23:43:25.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.067+0000 7f015b4bef40 1 -- 172.21.15.167:0/1556211171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b800 msgr2=0x560277c80580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.067+0000 7f015b4bef40 1 --2- 172.21.15.167:0/1556211171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b800 0x560277c80580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x560277af7e30 tx=0x560277a9f920 comp rx=0 tx=0).stop 2024-09-13T23:43:25.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.067+0000 7f015b4bef40 1 -- 172.21.15.167:0/1556211171 shutdown_connections 2024-09-13T23:43:25.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.067+0000 7f015b4bef40 1 --2- 172.21.15.167:0/1556211171 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560277ce7400 0x560277c81600 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.067+0000 7f015b4bef40 1 --2- 172.21.15.167:0/1556211171 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56027799b800 0x560277c80580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.067+0000 7f015b4bef40 1 -- 172.21.15.167:0/1556211171 >> 172.21.15.167:0/1556211171 conn(0x56027799ac00 msgr2=0x560277a9b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.067+0000 7f015b4bef40 1 -- 172.21.15.167:0/1556211171 shutdown_connections 2024-09-13T23:43:25.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.067+0000 7f015b4bef40 1 -- 172.21.15.167:0/1556211171 wait complete. 2024-09-13T23:43:25.080 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-13T23:43:25.081 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-09-13T23:43:25.081 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-09-13T23:43:25.081 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-09-13T23:43:25.116 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:25.116 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.107+0000 7f9d43728f40 1 Processor -- start 2024-09-13T23:43:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.107+0000 7f9d43728f40 1 -- start start 2024-09-13T23:43:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.107+0000 7f9d43728f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c96e575000 0x55c96e85a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.107+0000 7f9d43728f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c96e639500 con 0x55c96e575400 2024-09-13T23:43:25.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.107+0000 7f9d43728f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c96e602a00 con 0x55c96e575000 2024-09-13T23:43:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.107+0000 7f9d41ee1700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c96e575000 0x55c96e85a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.107+0000 7f9d416e0700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55c96e575400 0x55c96e674800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:40312/0 (socket says 172.21.15.167:40312) 2024-09-13T23:43:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.107+0000 7f9d416e0700 1 -- 172.21.15.167:0/1284564602 learned_addr learned my addr 172.21.15.167:0/1284564602 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d41ee1700 1 -- 172.21.15.167:0/1284564602 >> v1:172.21.15.12:6789/0 conn(0x55c96e575400 legacy=0x55c96e674800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d41ee1700 1 -- 172.21.15.167:0/1284564602 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c96e587d40 con 0x55c96e575000 2024-09-13T23:43:25.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d41ee1700 1 --2- 172.21.15.167:0/1284564602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55c96e855a70 tx=0x55c96e679920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acfade88608a90c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d40edf700 1 -- 172.21.15.167:0/1284564602 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96e6194a0 con 0x55c96e575000 2024-09-13T23:43:25.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d40edf700 1 -- 172.21.15.167:0/1284564602 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c96e639800 con 0x55c96e575000 2024-09-13T23:43:25.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d40edf700 1 -- 172.21.15.167:0/1284564602 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96e619600 con 0x55c96e575000 2024-09-13T23:43:25.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d43728f40 1 -- 172.21.15.167:0/1284564602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 msgr2=0x55c96e85a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d43728f40 1 --2- 172.21.15.167:0/1284564602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55c96e855a70 tx=0x55c96e679920 comp rx=0 tx=0).stop 2024-09-13T23:43:25.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d43728f40 1 -- 172.21.15.167:0/1284564602 shutdown_connections 2024-09-13T23:43:25.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d43728f40 1 --2- 172.21.15.167:0/1284564602 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d43728f40 1 -- 172.21.15.167:0/1284564602 >> 172.21.15.167:0/1284564602 conn(0x55c96e574c00 msgr2=0x55c96e674000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d43728f40 1 -- 172.21.15.167:0/1284564602 shutdown_connections 2024-09-13T23:43:25.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.111+0000 7f9d43728f40 1 -- 172.21.15.167:0/1284564602 wait complete. 2024-09-13T23:43:25.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 Processor -- start 2024-09-13T23:43:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 -- start start 2024-09-13T23:43:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96e602500 con 0x55c96e575000 2024-09-13T23:43:25.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d41ee1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d41ee1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43080/0 (socket says 172.21.15.167:43080) 2024-09-13T23:43:25.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3481428512 learned_addr learned my addr 172.21.15.167:0/3481428512 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3481428512 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c96e587ba0 con 0x55c96e575000 2024-09-13T23:43:25.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d41ee1700 1 --2- 172.21.15.167:0/3481428512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55c96e6d1d40 tx=0x55c96e679800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c890bec60df54b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d406de700 1 -- 172.21.15.167:0/3481428512 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96e619760 con 0x55c96e575000 2024-09-13T23:43:25.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d406de700 1 -- 172.21.15.167:0/3481428512 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c96e639b00 con 0x55c96e575000 2024-09-13T23:43:25.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d406de700 1 -- 172.21.15.167:0/3481428512 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96e6194a0 con 0x55c96e575000 2024-09-13T23:43:25.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 -- 172.21.15.167:0/3481428512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 msgr2=0x55c96e85a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 --2- 172.21.15.167:0/3481428512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55c96e6d1d40 tx=0x55c96e679800 comp rx=0 tx=0).stop 2024-09-13T23:43:25.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 -- 172.21.15.167:0/3481428512 shutdown_connections 2024-09-13T23:43:25.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 --2- 172.21.15.167:0/3481428512 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 -- 172.21.15.167:0/3481428512 >> 172.21.15.167:0/3481428512 conn(0x55c96e574c00 msgr2=0x55c96e675000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 -- 172.21.15.167:0/3481428512 shutdown_connections 2024-09-13T23:43:25.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 -- 172.21.15.167:0/3481428512 wait complete. 2024-09-13T23:43:25.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 Processor -- start 2024-09-13T23:43:25.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.115+0000 7f9d43728f40 1 -- start start 2024-09-13T23:43:25.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d43728f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d43728f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96e6023c0 con 0x55c96e575000 2024-09-13T23:43:25.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d41ee1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d41ee1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43094/0 (socket says 172.21.15.167:43094) 2024-09-13T23:43:25.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d41ee1700 1 -- 172.21.15.167:0/414447534 learned_addr learned my addr 172.21.15.167:0/414447534 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d41ee1700 1 -- 172.21.15.167:0/414447534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c96e587a00 con 0x55c96e575000 2024-09-13T23:43:25.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d41ee1700 1 --2- 172.21.15.167:0/414447534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55c96e6d16b0 tx=0x55c96e679920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d3eedb700 1 -- 172.21.15.167:0/414447534 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96e619600 con 0x55c96e575000 2024-09-13T23:43:25.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d3eedb700 1 -- 172.21.15.167:0/414447534 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c96e639c80 con 0x55c96e575000 2024-09-13T23:43:25.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d3eedb700 1 -- 172.21.15.167:0/414447534 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96e6198c0 con 0x55c96e575000 2024-09-13T23:43:25.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d43728f40 1 -- 172.21.15.167:0/414447534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c96e587860 con 0x55c96e575000 2024-09-13T23:43:25.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d43728f40 1 -- 172.21.15.167:0/414447534 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c96e5876c0 con 0x55c96e575000 2024-09-13T23:43:25.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d43728f40 1 Processor -- start 2024-09-13T23:43:25.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d43728f40 1 -- start start 2024-09-13T23:43:25.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d43728f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 0x55c96e85ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d43728f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96e59a640 con 0x55c96e575800 2024-09-13T23:43:25.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d426e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 0x55c96e85ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d426e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 0x55c96e85ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43096/0 (socket says 172.21.15.167:43096) 2024-09-13T23:43:25.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d426e2700 1 -- 172.21.15.167:0/2039672122 learned_addr learned my addr 172.21.15.167:0/2039672122 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.119+0000 7f9d3eedb700 1 -- 172.21.15.167:0/414447534 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c96e66eb00 con 0x55c96e575000 2024-09-13T23:43:25.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d426e2700 1 -- 172.21.15.167:0/2039672122 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c96e586000 con 0x55c96e575800 2024-09-13T23:43:25.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d3eedb700 1 --2- 172.21.15.167:0/414447534 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96e8aa000 0x55c96e85b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d3eedb700 1 -- 172.21.15.167:0/414447534 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c96e5db180 con 0x55c96e575000 2024-09-13T23:43:25.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d426e2700 1 --2- 172.21.15.167:0/2039672122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 0x55c96e85ab00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55c96e89cd20 tx=0x55c96e8e28a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b336edceeb6dd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d41ee1700 1 --2- 172.21.15.167:0/414447534 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96e8aa000 0x55c96e85b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d3c6d6700 1 -- 172.21.15.167:0/2039672122 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96e619600 con 0x55c96e575800 2024-09-13T23:43:25.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d3c6d6700 1 -- 172.21.15.167:0/2039672122 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c96e8e4000 con 0x55c96e575800 2024-09-13T23:43:25.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d3c6d6700 1 -- 172.21.15.167:0/2039672122 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96e6198c0 con 0x55c96e575800 2024-09-13T23:43:25.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d43728f40 1 -- 172.21.15.167:0/2039672122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 msgr2=0x55c96e85ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d41ee1700 1 --2- 172.21.15.167:0/414447534 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96e8aa000 0x55c96e85b080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55c96e89c870 tx=0x55c96e8e2120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d43728f40 1 --2- 172.21.15.167:0/2039672122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 0x55c96e85ab00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55c96e89cd20 tx=0x55c96e8e28a0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d43728f40 1 -- 172.21.15.167:0/2039672122 shutdown_connections 2024-09-13T23:43:25.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d43728f40 1 --2- 172.21.15.167:0/2039672122 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 0x55c96e85ab00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d43728f40 1 -- 172.21.15.167:0/2039672122 >> 172.21.15.167:0/2039672122 conn(0x55c96e575400 msgr2=0x55c96e675800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d43728f40 1 -- 172.21.15.167:0/2039672122 shutdown_connections 2024-09-13T23:43:25.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d43728f40 1 -- 172.21.15.167:0/2039672122 wait complete. 2024-09-13T23:43:25.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d43728f40 1 Processor -- start 2024-09-13T23:43:25.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d43728f40 1 -- start start 2024-09-13T23:43:25.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d43728f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 0x55c96e85ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d43728f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96e59a8c0 con 0x55c96e575800 2024-09-13T23:43:25.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d426e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 0x55c96e85ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d426e2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 0x55c96e85ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43098/0 (socket says 172.21.15.167:43098) 2024-09-13T23:43:25.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 learned_addr learned my addr 172.21.15.167:0/3152406271 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c96e586ea0 con 0x55c96e575800 2024-09-13T23:43:25.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.123+0000 7f9d426e2700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 0x55c96e85ab00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55c96e9360f0 tx=0x55c96e679b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d3aed3700 1 -- 172.21.15.167:0/3152406271 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96e619760 con 0x55c96e575800 2024-09-13T23:43:25.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c96e586820 con 0x55c96e575800 2024-09-13T23:43:25.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c96e5869c0 con 0x55c96e575800 2024-09-13T23:43:25.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d3aed3700 1 -- 172.21.15.167:0/3152406271 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c96e8e4180 con 0x55c96e575800 2024-09-13T23:43:25.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d3aed3700 1 -- 172.21.15.167:0/3152406271 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96e619600 con 0x55c96e575800 2024-09-13T23:43:25.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d3aed3700 1 -- 172.21.15.167:0/3152406271 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c96e66edc0 con 0x55c96e575800 2024-09-13T23:43:25.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d3aed3700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96e8aac00 0x55c96e85b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d3aed3700 1 -- 172.21.15.167:0/3152406271 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c96e5db340 con 0x55c96e575800 2024-09-13T23:43:25.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d416e0700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96e8aac00 0x55c96e85b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c96e8ab000 0x55c96e85bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab400 con 0x55c96e8ab000 2024-09-13T23:43:25.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d426e2700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c96e8ab000 0x55c96e85bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d416e0700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96e8aac00 0x55c96e85b600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55c96e937dd0 tx=0x55c96e8e3f80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d426e2700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c96e8ab000 0x55c96e85bb80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 20 realm 2024-09-13T23:43:25.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab000 2024-09-13T23:43:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.127+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8abc00 con 0x55c96e8ab000 2024-09-13T23:43:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d43728f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d43728f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d43728f40 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c96e8ab400 0x55c96e85c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab400 2024-09-13T23:43:25.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d41ee1700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c96e8ab400 0x55c96e85c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d41ee1700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c96e8ab400 0x55c96e85c100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d43728f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:25.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d43728f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8abc00 con 0x55c96e8ab400 2024-09-13T23:43:25.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d43728f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:25.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d43728f40 20 searching for the correct realm 2024-09-13T23:43:25.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d43728f40 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c96e9ea000 0x55c96e85c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ea400 con 0x55c96e9ea000 2024-09-13T23:43:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d416e0700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c96e9ea000 0x55c96e85c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d416e0700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c96e9ea000 0x55c96e85c680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.131+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ea400 con 0x55c96e8ab400 2024-09-13T23:43:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d41ee1700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c96e9ea400 0x55c96e85cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea400 2024-09-13T23:43:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d426e2700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c96e9ea400 0x55c96e85cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d426e2700 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c96e9ea400 0x55c96e85cc00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab000 2024-09-13T23:43:25.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab000 2024-09-13T23:43:25.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea000 2024-09-13T23:43:25.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab000 2024-09-13T23:43:25.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.135+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab400 2024-09-13T23:43:25.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab400 2024-09-13T23:43:25.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab000 2024-09-13T23:43:25.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea400 2024-09-13T23:43:25.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab000 2024-09-13T23:43:25.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea400 2024-09-13T23:43:25.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab400 2024-09-13T23:43:25.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea000 2024-09-13T23:43:25.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab000 2024-09-13T23:43:25.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.139+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab400 2024-09-13T23:43:25.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea000 2024-09-13T23:43:25.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab000 2024-09-13T23:43:25.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab000 2024-09-13T23:43:25.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab000 2024-09-13T23:43:25.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea000 2024-09-13T23:43:25.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea400 2024-09-13T23:43:25.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab000 2024-09-13T23:43:25.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.143+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea400 2024-09-13T23:43:25.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.147+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.147+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea000 2024-09-13T23:43:25.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.147+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.147+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea400 2024-09-13T23:43:25.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.147+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.147+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab400 2024-09-13T23:43:25.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.147+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.147+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab400 2024-09-13T23:43:25.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.147+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.147+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea000 2024-09-13T23:43:25.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.147+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.147+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab400 2024-09-13T23:43:25.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e8ab400 2024-09-13T23:43:25.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:25.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:25.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:25.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:25.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ea800 con 0x55c96e8ab000 2024-09-13T23:43:25.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eac00 con 0x55c96e8ab400 2024-09-13T23:43:25.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:25.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb000 con 0x55c96e8ab400 2024-09-13T23:43:25.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:25.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 zone default found 2024-09-13T23:43:25.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 4 Realm: () 2024-09-13T23:43:25.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:25.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:25.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:25.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 zonegroup default 2024-09-13T23:43:25.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb400 con 0x55c96e9ea400 2024-09-13T23:43:25.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.151+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb800 con 0x55c96e9ea000 2024-09-13T23:43:25.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.155+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.155+0000 7f9d43728f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.155+0000 7f9d43728f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.155+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ebc00 con 0x55c96e9ea000 2024-09-13T23:43:25.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.155+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.155+0000 7f9d43728f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.155+0000 7f9d43728f40 20 started sync module instance, tier type = 2024-09-13T23:43:25.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.155+0000 7f9d43728f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:25.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.155+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb800 con 0x55c96e8ab400 2024-09-13T23:43:25.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.155+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.155+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94323631004416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ebc00 con 0x55c96e8ab400 2024-09-13T23:43:25.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.159+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eac00 con 0x55c96e8ab000 2024-09-13T23:43:25.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.159+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94323631004416] v33'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.159+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.159+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94323631005568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb000 con 0x55c96e8ab000 2024-09-13T23:43:25.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.159+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb800 con 0x55c96e8ab400 2024-09-13T23:43:25.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.163+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94323631005568] v33'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.163+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.163+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94323631006720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8abc00 con 0x55c96e8ab400 2024-09-13T23:43:25.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.163+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8ab800 con 0x55c96e9ea400 2024-09-13T23:43:25.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.167+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.167+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94323631007872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ea800 con 0x55c96e9ea400 2024-09-13T23:43:25.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.167+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb400 con 0x55c96e8ab400 2024-09-13T23:43:25.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.167+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94323631006720] v33'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.167+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94323631007872] v33'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.167+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.167+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94323631009024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eac00 con 0x55c96e8ab400 2024-09-13T23:43:25.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.167+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb000 con 0x55c96e9ea000 2024-09-13T23:43:25.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.171+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.171+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94323631010176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ec000 con 0x55c96e9ea000 2024-09-13T23:43:25.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.171+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ec400 con 0x55c96e9ea000 2024-09-13T23:43:25.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.171+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94323631009024] v33'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.171+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94323631010176] v33'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.171+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.171+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94323631011328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ec800 con 0x55c96e9ea000 2024-09-13T23:43:25.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.171+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ecc00 con 0x55c96e9ea400 2024-09-13T23:43:25.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'12 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94323631012480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ed000 con 0x55c96e9ea400 2024-09-13T23:43:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d43728f40 20 add_watcher() i=0 2024-09-13T23:43:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d43728f40 20 add_watcher() i=1 2024-09-13T23:43:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d43728f40 20 add_watcher() i=2 2024-09-13T23:43:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d43728f40 20 add_watcher() i=3 2024-09-13T23:43:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d43728f40 20 add_watcher() i=4 2024-09-13T23:43:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d43728f40 20 add_watcher() i=5 2024-09-13T23:43:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94323631011328] v33'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d43728f40 20 add_watcher() i=6 2024-09-13T23:43:25.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94323631012480] v33'13 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d43728f40 20 add_watcher() i=7 2024-09-13T23:43:25.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d43728f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:25.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ed400 con 0x55c96e8ab000 2024-09-13T23:43:25.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.175+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94323632709632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ed800 con 0x55c96e8ab000 2024-09-13T23:43:25.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94323632709632] v33'36 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d28eaf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d43728f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d43728f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:25.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d43728f40 1 Processor -- start 2024-09-13T23:43:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d43728f40 1 -- start start 2024-09-13T23:43:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d43728f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 0x55c96e85d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d43728f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96e6023c0 con 0x55c96e9ecc00 2024-09-13T23:43:25.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d416e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 0x55c96e85d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d416e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 0x55c96e85d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43104/0 (socket says 172.21.15.167:43104) 2024-09-13T23:43:25.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d416e0700 1 -- 172.21.15.167:0/1143631700 learned_addr learned my addr 172.21.15.167:0/1143631700 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d416e0700 1 -- 172.21.15.167:0/1143631700 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c96e586b60 con 0x55c96e9ecc00 2024-09-13T23:43:25.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d416e0700 1 --2- 172.21.15.167:0/1143631700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 0x55c96e85d180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55c96e9fc090 tx=0x55c96e9fe000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec63e1df719088b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d286ae700 1 -- 172.21.15.167:0/1143631700 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96ea06000 con 0x55c96e9ecc00 2024-09-13T23:43:25.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d286ae700 1 -- 172.21.15.167:0/1143631700 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c96e8e4600 con 0x55c96e9ecc00 2024-09-13T23:43:25.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.179+0000 7f9d286ae700 1 -- 172.21.15.167:0/1143631700 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96ea06160 con 0x55c96e9ecc00 2024-09-13T23:43:25.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 -- 172.21.15.167:0/1143631700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 msgr2=0x55c96e85d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 --2- 172.21.15.167:0/1143631700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 0x55c96e85d180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55c96e9fc090 tx=0x55c96e9fe000 comp rx=0 tx=0).stop 2024-09-13T23:43:25.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 -- 172.21.15.167:0/1143631700 shutdown_connections 2024-09-13T23:43:25.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 --2- 172.21.15.167:0/1143631700 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 0x55c96e85d180 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 -- 172.21.15.167:0/1143631700 >> 172.21.15.167:0/1143631700 conn(0x55c96e9edc00 msgr2=0x55c96e676000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 -- 172.21.15.167:0/1143631700 shutdown_connections 2024-09-13T23:43:25.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 -- 172.21.15.167:0/1143631700 wait complete. 2024-09-13T23:43:25.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 Processor -- start 2024-09-13T23:43:25.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 -- start start 2024-09-13T23:43:25.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 0x55c96e85d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c96e602500 con 0x55c96e9ecc00 2024-09-13T23:43:25.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d416e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 0x55c96e85d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d416e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 0x55c96e85d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43116/0 (socket says 172.21.15.167:43116) 2024-09-13T23:43:25.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d416e0700 1 -- 172.21.15.167:0/4046154723 learned_addr learned my addr 172.21.15.167:0/4046154723 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d416e0700 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c96e586d00 con 0x55c96e9ecc00 2024-09-13T23:43:25.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d416e0700 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 0x55c96e85d180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55c96e9fd350 tx=0x55c96e9fe2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d26eab700 1 -- 172.21.15.167:0/4046154723 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96ea062c0 con 0x55c96e9ecc00 2024-09-13T23:43:25.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d26eab700 1 -- 172.21.15.167:0/4046154723 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c96e8e4300 con 0x55c96e9ecc00 2024-09-13T23:43:25.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c96e586340 con 0x55c96e9ecc00 2024-09-13T23:43:25.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d26eab700 1 -- 172.21.15.167:0/4046154723 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c96ea06000 con 0x55c96e9ecc00 2024-09-13T23:43:25.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.183+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c96e586b60 con 0x55c96e9ecc00 2024-09-13T23:43:25.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.187+0000 7f9d26eab700 1 -- 172.21.15.167:0/4046154723 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c96e66fb80 con 0x55c96e9ecc00 2024-09-13T23:43:25.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.187+0000 7f9d26eab700 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96ea82000 0x55c96ea88000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.187+0000 7f9d26eab700 1 -- 172.21.15.167:0/4046154723 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c96e5db500 con 0x55c96e9ecc00 2024-09-13T23:43:25.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.187+0000 7f9d426e2700 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96ea82000 0x55c96ea88000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.187+0000 7f9d43728f40 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c96e9ed000 0x55c96e85d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.187+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ed400 con 0x55c96e9ed000 2024-09-13T23:43:25.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.187+0000 7f9d41ee1700 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c96e9ed000 0x55c96e85d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.187+0000 7f9d426e2700 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96ea82000 0x55c96ea88000 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55c96e89cb10 tx=0x55c96e8e33e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.187+0000 7f9d41ee1700 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c96e9ed000 0x55c96e85d700 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.191+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.191+0000 7f9d43728f40 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c96e9ec000 0x55c96ea88580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.191+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ec400 con 0x55c96e9ec000 2024-09-13T23:43:25.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.191+0000 7f9d416e0700 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c96e9ec000 0x55c96ea88580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.191+0000 7f9d416e0700 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c96e9ec000 0x55c96ea88580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.191+0000 7f9d416e0700 1 -- 172.21.15.167:0/4046154723 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'12 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ec000 2024-09-13T23:43:25.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.191+0000 7f9d43728f40 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c96e9ea800 0x55c96ea88b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.191+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb000 con 0x55c96e9ea800 2024-09-13T23:43:25.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.195+0000 7f9d426e2700 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c96e9ea800 0x55c96ea88b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.195+0000 7f9d426e2700 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c96e9ea800 0x55c96ea88b00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.195+0000 7f9d426e2700 1 -- 172.21.15.167:0/4046154723 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'12 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea800 2024-09-13T23:43:25.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.195+0000 7f9d43728f40 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c96e8ab800 0x55c96ea89080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.195+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea82400 con 0x55c96e8ab800 2024-09-13T23:43:25.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.195+0000 7f9d41ee1700 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c96e8ab800 0x55c96ea89080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.195+0000 7f9d41ee1700 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c96e8ab800 0x55c96ea89080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.199+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'37 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab800 2024-09-13T23:43:25.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.199+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea82800 con 0x55c96e9ed000 2024-09-13T23:43:25.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.203+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'26 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.203+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea82c00 con 0x55c96e9ea800 2024-09-13T23:43:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.203+0000 7f9d426e2700 1 -- 172.21.15.167:0/4046154723 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'17 uv10 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea800 2024-09-13T23:43:25.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.203+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea83000 con 0x55c96e9ed000 2024-09-13T23:43:25.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.207+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'14 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.207+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea83400 con 0x55c96e8ab800 2024-09-13T23:43:25.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.207+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'17 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab800 2024-09-13T23:43:25.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.207+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea83800 con 0x55c96e9ed000 2024-09-13T23:43:25.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.211+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.211+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea83c00 con 0x55c96e9ed000 2024-09-13T23:43:25.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.215+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'18 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.215+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ed400 con 0x55c96e9ed000 2024-09-13T23:43:25.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.215+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'27 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.215+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ebc00 con 0x55c96e9ea800 2024-09-13T23:43:25.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.219+0000 7f9d426e2700 1 -- 172.21.15.167:0/4046154723 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'14 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea800 2024-09-13T23:43:25.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.219+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb800 con 0x55c96e9ed000 2024-09-13T23:43:25.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.223+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.223+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb400 con 0x55c96e8ab800 2024-09-13T23:43:25.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.223+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'23 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab800 2024-09-13T23:43:25.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.223+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eac00 con 0x55c96e8ab800 2024-09-13T23:43:25.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.227+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'38 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab800 2024-09-13T23:43:25.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.227+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8abc00 con 0x55c96e9ed000 2024-09-13T23:43:25.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.231+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.231+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ed400 con 0x55c96e9ed000 2024-09-13T23:43:25.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.231+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'28 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.231+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea82400 con 0x55c96e9ea800 2024-09-13T23:43:25.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.235+0000 7f9d426e2700 1 -- 172.21.15.167:0/4046154723 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'12 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea800 2024-09-13T23:43:25.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.235+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea82800 con 0x55c96e9ed000 2024-09-13T23:43:25.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.239+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'20 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.239+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea83000 con 0x55c96e8ab800 2024-09-13T23:43:25.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.239+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'39 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab800 2024-09-13T23:43:25.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.239+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea83400 con 0x55c96e9ed000 2024-09-13T23:43:25.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.243+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'21 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.243+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea83800 con 0x55c96e9ed000 2024-09-13T23:43:25.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.247+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'22 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.247+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea83c00 con 0x55c96e9ea800 2024-09-13T23:43:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.247+0000 7f9d426e2700 1 -- 172.21.15.167:0/4046154723 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea800 2024-09-13T23:43:25.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.247+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea82c00 con 0x55c96e9ed000 2024-09-13T23:43:25.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.251+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.251+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea82400 con 0x55c96e9ea800 2024-09-13T23:43:25.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.255+0000 7f9d426e2700 1 -- 172.21.15.167:0/4046154723 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'8 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea800 2024-09-13T23:43:25.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.255+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea83c00 con 0x55c96e9ea800 2024-09-13T23:43:25.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.259+0000 7f9d426e2700 1 -- 172.21.15.167:0/4046154723 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'18 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea800 2024-09-13T23:43:25.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.259+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea82800 con 0x55c96e9ea800 2024-09-13T23:43:25.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.259+0000 7f9d426e2700 1 -- 172.21.15.167:0/4046154723 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea800 2024-09-13T23:43:25.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.259+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea83000 con 0x55c96e9ed000 2024-09-13T23:43:25.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.263+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'12 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.263+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96ea83400 con 0x55c96e8ab800 2024-09-13T23:43:25.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.267+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'18 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab800 2024-09-13T23:43:25.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.267+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ed800 con 0x55c96e9ed000 2024-09-13T23:43:25.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.267+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'22 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e9ed000 2024-09-13T23:43:25.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.267+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9ed400 con 0x55c96e8ab800 2024-09-13T23:43:25.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.271+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'24 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab800 2024-09-13T23:43:25.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.271+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e8abc00 con 0x55c96e9ec000 2024-09-13T23:43:25.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.271+0000 7f9d416e0700 1 -- 172.21.15.167:0/4046154723 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ec000 2024-09-13T23:43:25.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.275+0000 7f9d246a6700 20 reqs_thread_entry: start 2024-09-13T23:43:25.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.275+0000 7f9d43728f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:25.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.275+0000 7f9d43728f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:25.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.275+0000 7f9d226a2700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:55 2024 2024-09-13T23:43:25.286 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:25.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.275+0000 7f9d21e80700 20 reqs_thread_entry: start 2024-09-13T23:43:25.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.275+0000 7f9d43728f40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-13T23:43:25.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.275+0000 7f9d43728f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.275+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb800 con 0x55c96e9ea400 2024-09-13T23:43:25.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.279+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.279+0000 7f9d43728f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.279+0000 7f9d43728f40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-09-13T23:43:25.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.279+0000 7f9d43728f40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:43:25.287 INFO:teuthology.orchestra.run.smithi167.stderr:could not remove user: unable to remove user, user does not exist 2024-09-13T23:43:25.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.279+0000 7f9d28eaf700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:25.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.279+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94323632709632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eac00 con 0x55c96e8ab000 2024-09-13T23:43:25.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.283+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94323632709632] v33'40 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.283+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94323631004416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e9eb400 con 0x55c96e8ab400 2024-09-13T23:43:25.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.283+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94323631004416] v33'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.283+0000 7f9d43728f40 20 remove_watcher() i=0 2024-09-13T23:43:25.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.283+0000 7f9d43728f40 2 removed watcher, disabling cache 2024-09-13T23:43:25.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.283+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94323631005568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e946000 con 0x55c96e8ab000 2024-09-13T23:43:25.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.287+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94323631005568] v33'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e8ab000 2024-09-13T23:43:25.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.287+0000 7f9d43728f40 20 remove_watcher() i=1 2024-09-13T23:43:25.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.287+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94323631006720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e946400 con 0x55c96e8ab400 2024-09-13T23:43:25.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.287+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94323631006720] v33'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.287+0000 7f9d43728f40 20 remove_watcher() i=2 2024-09-13T23:43:25.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.287+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94323631007872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e946800 con 0x55c96e9ea400 2024-09-13T23:43:25.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.291+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94323631007872] v33'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.291+0000 7f9d43728f40 20 remove_watcher() i=3 2024-09-13T23:43:25.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.291+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94323631009024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e946c00 con 0x55c96e8ab400 2024-09-13T23:43:25.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.291+0000 7f9d41ee1700 1 -- 172.21.15.167:0/3152406271 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94323631009024] v33'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3200 con 0x55c96e8ab400 2024-09-13T23:43:25.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.295+0000 7f9d43728f40 20 remove_watcher() i=4 2024-09-13T23:43:25.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.295+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94323631010176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e947000 con 0x55c96e9ea000 2024-09-13T23:43:25.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.295+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94323631010176] v33'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.295+0000 7f9d43728f40 20 remove_watcher() i=5 2024-09-13T23:43:25.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.295+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94323631011328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e947400 con 0x55c96e9ea000 2024-09-13T23:43:25.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94323631011328] v33'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c3440 con 0x55c96e9ea000 2024-09-13T23:43:25.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 20 remove_watcher() i=6 2024-09-13T23:43:25.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94323631012480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c96e947800 con 0x55c96e9ea400 2024-09-13T23:43:25.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d426e2700 1 -- 172.21.15.167:0/3152406271 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94323631012480] v33'14 uv12 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c96e9c2b40 con 0x55c96e9ea400 2024-09-13T23:43:25.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 20 remove_watcher() i=7 2024-09-13T23:43:25.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c96e8ab400 msgr2=0x55c96e85c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c96e8ab400 0x55c96e85c100 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c96e8ab000 msgr2=0x55c96e85bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c96e8ab000 0x55c96e85bb80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c96e9ea400 msgr2=0x55c96e85cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c96e9ea400 0x55c96e85cc00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c96e9ea000 msgr2=0x55c96e85c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c96e9ea000 0x55c96e85c680 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96e8aac00 msgr2=0x55c96e85b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96e8aac00 0x55c96e85b600 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x55c96e937dd0 tx=0x55c96e8e3f80 comp rx=0 tx=0).stop 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 msgr2=0x55c96e85ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 0x55c96e85ab00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55c96e9360f0 tx=0x55c96e679b60 comp rx=0 tx=0).stop 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d416e0700 1 -- 172.21.15.167:0/3152406271 reap_dead start 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 shutdown_connections 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 --2- 172.21.15.167:0/3152406271 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575800 0x55c96e85ab00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 >> 172.21.15.167:0/3152406271 conn(0x55c96e575400 msgr2=0x55c96e675800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 shutdown_connections 2024-09-13T23:43:25.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.299+0000 7f9d43728f40 1 -- 172.21.15.167:0/3152406271 wait complete. 2024-09-13T23:43:25.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c96e9ed000 msgr2=0x55c96e85d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c96e9ed000 0x55c96e85d700 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c96e8ab800 msgr2=0x55c96ea89080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c96e8ab800 0x55c96ea89080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c96e9ec000 msgr2=0x55c96ea88580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c96e9ec000 0x55c96ea88580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c96e9ea800 msgr2=0x55c96ea88b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c96e9ea800 0x55c96ea88b00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96ea82000 msgr2=0x55c96ea88000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96ea82000 0x55c96ea88000 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55c96e89cb10 tx=0x55c96e8e33e0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d41ee1700 1 -- 172.21.15.167:0/4046154723 reap_dead start 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 msgr2=0x55c96e85d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 0x55c96e85d180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55c96e9fd350 tx=0x55c96e9fe2a0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 shutdown_connections 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 --2- 172.21.15.167:0/4046154723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e9ecc00 0x55c96e85d180 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 >> 172.21.15.167:0/4046154723 conn(0x55c96e9edc00 msgr2=0x55c96e676000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 shutdown_connections 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/4046154723 wait complete. 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/414447534 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96e8aa000 msgr2=0x55c96e85b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 --2- 172.21.15.167:0/414447534 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96e8aa000 0x55c96e85b080 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55c96e89c870 tx=0x55c96e8e2120 comp rx=0 tx=0).stop 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/414447534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 msgr2=0x55c96e85a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 --2- 172.21.15.167:0/414447534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55c96e6d16b0 tx=0x55c96e679920 comp rx=0 tx=0).stop 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/414447534 shutdown_connections 2024-09-13T23:43:25.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 --2- 172.21.15.167:0/414447534 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c96e8aa000 0x55c96e85b080 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 --2- 172.21.15.167:0/414447534 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c96e575000 0x55c96e85a580 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/414447534 >> 172.21.15.167:0/414447534 conn(0x55c96e574c00 msgr2=0x55c96e675000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/414447534 shutdown_connections 2024-09-13T23:43:25.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.303+0000 7f9d43728f40 1 -- 172.21.15.167:0/414447534 wait complete. 2024-09-13T23:43:25.317 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-13T23:43:25.318 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-09-13T23:43:25.318 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-09-13T23:43:25.318 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-09-13T23:43:25.360 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:25.360 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:25.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.351+0000 7f84fa86cf40 1 Processor -- start 2024-09-13T23:43:25.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.351+0000 7f84fa86cf40 1 -- start start 2024-09-13T23:43:25.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.351+0000 7f84fa86cf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5623babb9400 0x5623bae9e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.351+0000 7f84fa86cf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5623bac7d500 con 0x5623babb9000 2024-09-13T23:43:25.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.351+0000 7f84fa86cf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5623bac46a00 con 0x5623babb9400 2024-09-13T23:43:25.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.351+0000 7f84f8824700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5623babb9400 0x5623bae9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.351+0000 7f84f8824700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5623babb9400 0x5623bae9e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43122/0 (socket says 172.21.15.167:43122) 2024-09-13T23:43:25.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.351+0000 7f84f8824700 1 -- 172.21.15.167:0/4237262035 learned_addr learned my addr 172.21.15.167:0/4237262035 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.351+0000 7f84f9025700 1 --1- 172.21.15.167:0/4237262035 >> v1:172.21.15.12:6789/0 conn(0x5623babb9000 0x5623bacb8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:40324/0 (socket says 172.21.15.167:40324) 2024-09-13T23:43:25.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84f8824700 1 -- 172.21.15.167:0/4237262035 >> v1:172.21.15.12:6789/0 conn(0x5623babb9000 legacy=0x5623bacb8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84f8824700 1 -- 172.21.15.167:0/4237262035 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623babcbd40 con 0x5623babb9400 2024-09-13T23:43:25.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84f8824700 1 --2- 172.21.15.167:0/4237262035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9400 0x5623bae9e580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5623bae99920 tx=0x5623bacbd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8238f1c0c82fc18 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84f8023700 1 -- 172.21.15.167:0/4237262035 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bac5d4a0 con 0x5623babb9400 2024-09-13T23:43:25.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84f8023700 1 -- 172.21.15.167:0/4237262035 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623bac7d800 con 0x5623babb9400 2024-09-13T23:43:25.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84f8023700 1 -- 172.21.15.167:0/4237262035 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bac5d600 con 0x5623babb9400 2024-09-13T23:43:25.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84fa86cf40 1 -- 172.21.15.167:0/4237262035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9400 msgr2=0x5623bae9e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/4237262035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9400 0x5623bae9e580 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5623bae99920 tx=0x5623bacbd920 comp rx=0 tx=0).stop 2024-09-13T23:43:25.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84fa86cf40 1 -- 172.21.15.167:0/4237262035 shutdown_connections 2024-09-13T23:43:25.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/4237262035 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9400 0x5623bae9e580 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84fa86cf40 1 -- 172.21.15.167:0/4237262035 >> 172.21.15.167:0/4237262035 conn(0x5623babb8c00 msgr2=0x5623bacb8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84fa86cf40 1 -- 172.21.15.167:0/4237262035 shutdown_connections 2024-09-13T23:43:25.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84fa86cf40 1 -- 172.21.15.167:0/4237262035 wait complete. 2024-09-13T23:43:25.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84fa86cf40 1 Processor -- start 2024-09-13T23:43:25.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84fa86cf40 1 -- start start 2024-09-13T23:43:25.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84fa86cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 0x5623bae9e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84fa86cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bac46500 con 0x5623babb9000 2024-09-13T23:43:25.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84f9025700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 0x5623bae9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84f9025700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 0x5623bae9e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43138/0 (socket says 172.21.15.167:43138) 2024-09-13T23:43:25.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84f9025700 1 -- 172.21.15.167:0/1963103269 learned_addr learned my addr 172.21.15.167:0/1963103269 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84f9025700 1 -- 172.21.15.167:0/1963103269 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623babcbba0 con 0x5623babb9000 2024-09-13T23:43:25.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.355+0000 7f84f9025700 1 --2- 172.21.15.167:0/1963103269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 0x5623bae9e580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5623bad15e90 tx=0x5623bacbdb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86071cfb61e52693 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84f7822700 1 -- 172.21.15.167:0/1963103269 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bac5d4a0 con 0x5623babb9000 2024-09-13T23:43:25.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84f7822700 1 -- 172.21.15.167:0/1963103269 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623bac7db00 con 0x5623babb9000 2024-09-13T23:43:25.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84f7822700 1 -- 172.21.15.167:0/1963103269 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bac5d600 con 0x5623babb9000 2024-09-13T23:43:25.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84fa86cf40 1 -- 172.21.15.167:0/1963103269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 msgr2=0x5623bae9e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/1963103269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 0x5623bae9e580 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x5623bad15e90 tx=0x5623bacbdb60 comp rx=0 tx=0).stop 2024-09-13T23:43:25.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84fa86cf40 1 -- 172.21.15.167:0/1963103269 shutdown_connections 2024-09-13T23:43:25.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/1963103269 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 0x5623bae9e580 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84fa86cf40 1 -- 172.21.15.167:0/1963103269 >> 172.21.15.167:0/1963103269 conn(0x5623babb8c00 msgr2=0x5623bacb9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84fa86cf40 1 -- 172.21.15.167:0/1963103269 shutdown_connections 2024-09-13T23:43:25.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84fa86cf40 1 -- 172.21.15.167:0/1963103269 wait complete. 2024-09-13T23:43:25.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84fa86cf40 1 Processor -- start 2024-09-13T23:43:25.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84fa86cf40 1 -- start start 2024-09-13T23:43:25.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84fa86cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 0x5623bae9e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84fa86cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bac463c0 con 0x5623babb9000 2024-09-13T23:43:25.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84f9025700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 0x5623bae9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84f9025700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 0x5623bae9e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43144/0 (socket says 172.21.15.167:43144) 2024-09-13T23:43:25.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84f9025700 1 -- 172.21.15.167:0/3834612799 learned_addr learned my addr 172.21.15.167:0/3834612799 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84f9025700 1 -- 172.21.15.167:0/3834612799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623babcba00 con 0x5623babb9000 2024-09-13T23:43:25.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84f9025700 1 --2- 172.21.15.167:0/3834612799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 0x5623bae9e580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5623bba2a0f0 tx=0x5623bacbda40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84f601f700 1 -- 172.21.15.167:0/3834612799 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bac5d8c0 con 0x5623babb9000 2024-09-13T23:43:25.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.359+0000 7f84f601f700 1 -- 172.21.15.167:0/3834612799 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623bac7dc80 con 0x5623babb9000 2024-09-13T23:43:25.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84fa86cf40 1 -- 172.21.15.167:0/3834612799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5623babcb860 con 0x5623babb9000 2024-09-13T23:43:25.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f601f700 1 -- 172.21.15.167:0/3834612799 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bac5d4a0 con 0x5623babb9000 2024-09-13T23:43:25.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84fa86cf40 1 -- 172.21.15.167:0/3834612799 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623babcb6c0 con 0x5623babb9000 2024-09-13T23:43:25.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84fa86cf40 1 Processor -- start 2024-09-13T23:43:25.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84fa86cf40 1 -- start start 2024-09-13T23:43:25.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84fa86cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 0x5623bae9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84fa86cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623babde640 con 0x5623babb9800 2024-09-13T23:43:25.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f9826700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 0x5623bae9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f601f700 1 -- 172.21.15.167:0/3834612799 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5623bacb2b00 con 0x5623babb9000 2024-09-13T23:43:25.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f9826700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 0x5623bae9eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43154/0 (socket says 172.21.15.167:43154) 2024-09-13T23:43:25.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f9826700 1 -- 172.21.15.167:0/1987581953 learned_addr learned my addr 172.21.15.167:0/1987581953 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f9826700 1 -- 172.21.15.167:0/1987581953 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623babca000 con 0x5623babb9800 2024-09-13T23:43:25.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f601f700 1 --2- 172.21.15.167:0/3834612799 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623baf6e000 0x5623bae9f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f601f700 1 -- 172.21.15.167:0/3834612799 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x5623bac1f180 con 0x5623babb9000 2024-09-13T23:43:25.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f9826700 1 --2- 172.21.15.167:0/1987581953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 0x5623bae9eb00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5623bba2b6b0 tx=0x5623baf20960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24be91bc23d69d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f9025700 1 --2- 172.21.15.167:0/3834612799 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623baf6e000 0x5623bae9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f381a700 1 -- 172.21.15.167:0/1987581953 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bac5d8c0 con 0x5623babb9800 2024-09-13T23:43:25.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f381a700 1 -- 172.21.15.167:0/1987581953 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623baf12000 con 0x5623babb9800 2024-09-13T23:43:25.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.363+0000 7f84f381a700 1 -- 172.21.15.167:0/1987581953 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bac5d4a0 con 0x5623babb9800 2024-09-13T23:43:25.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84fa86cf40 1 -- 172.21.15.167:0/1987581953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 msgr2=0x5623bae9eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84f9025700 1 --2- 172.21.15.167:0/3834612799 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623baf6e000 0x5623bae9f080 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x5623baf4a660 tx=0x5623baf20240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/1987581953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 0x5623bae9eb00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x5623bba2b6b0 tx=0x5623baf20960 comp rx=0 tx=0).stop 2024-09-13T23:43:25.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84fa86cf40 1 -- 172.21.15.167:0/1987581953 shutdown_connections 2024-09-13T23:43:25.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/1987581953 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 0x5623bae9eb00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84fa86cf40 1 -- 172.21.15.167:0/1987581953 >> 172.21.15.167:0/1987581953 conn(0x5623babb9400 msgr2=0x5623bacb9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84fa86cf40 1 -- 172.21.15.167:0/1987581953 shutdown_connections 2024-09-13T23:43:25.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84fa86cf40 1 -- 172.21.15.167:0/1987581953 wait complete. 2024-09-13T23:43:25.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84fa86cf40 1 Processor -- start 2024-09-13T23:43:25.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84fa86cf40 1 -- start start 2024-09-13T23:43:25.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84fa86cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 0x5623bae9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84fa86cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623babde8c0 con 0x5623babb9800 2024-09-13T23:43:25.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84f9826700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 0x5623bae9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84f9826700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 0x5623bae9eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43160/0 (socket says 172.21.15.167:43160) 2024-09-13T23:43:25.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 learned_addr learned my addr 172.21.15.167:0/349458305 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623babcaea0 con 0x5623babb9800 2024-09-13T23:43:25.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84f9826700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 0x5623bae9eb00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5623bba2a3f0 tx=0x5623baf202a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84f2017700 1 -- 172.21.15.167:0/349458305 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bac5db80 con 0x5623babb9800 2024-09-13T23:43:25.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84f2017700 1 -- 172.21.15.167:0/349458305 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623baf12180 con 0x5623babb9800 2024-09-13T23:43:25.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5623babca820 con 0x5623babb9800 2024-09-13T23:43:25.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.367+0000 7f84f2017700 1 -- 172.21.15.167:0/349458305 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bac5d8c0 con 0x5623babb9800 2024-09-13T23:43:25.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623babca9c0 con 0x5623babb9800 2024-09-13T23:43:25.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84f2017700 1 -- 172.21.15.167:0/349458305 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5623bacb2dc0 con 0x5623babb9800 2024-09-13T23:43:25.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84f2017700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623baf6ec00 0x5623bae9f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84f2017700 1 -- 172.21.15.167:0/349458305 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x5623bac1f340 con 0x5623babb9800 2024-09-13T23:43:25.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84fa86cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84f8824700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623baf6ec00 0x5623bae9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5623baf6f000 0x5623bae9fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f400 con 0x5623baf6f000 2024-09-13T23:43:25.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84f9826700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5623baf6f000 0x5623bae9fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84f8824700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623baf6ec00 0x5623bae9f600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5623bae99920 tx=0x5623bacbd800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84f9826700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5623baf6f000 0x5623bae9fb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84fa86cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84fa86cf40 20 realm 2024-09-13T23:43:25.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84fa86cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.371+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f000 2024-09-13T23:43:25.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:25.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6fc00 con 0x5623baf6f000 2024-09-13T23:43:25.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5623baf6f400 0x5623baea0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f400 2024-09-13T23:43:25.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84f9025700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5623baf6f400 0x5623baea0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84f9025700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5623baf6f400 0x5623baea0100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:25.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6fc00 con 0x5623baf6f400 2024-09-13T23:43:25.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:25.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 20 searching for the correct realm 2024-09-13T23:43:25.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5623bb01c000 0x5623baea0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01c400 con 0x5623bb01c000 2024-09-13T23:43:25.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.375+0000 7f84f8824700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5623bb01c000 0x5623baea0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f8824700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5623bb01c000 0x5623baea0680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01c400 con 0x5623baf6f400 2024-09-13T23:43:25.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f9025700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5623bb01c400 0x5623baea0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c400 2024-09-13T23:43:25.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f9826700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5623bb01c400 0x5623baea0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f9826700 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5623bb01c400 0x5623baea0c00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f000 2024-09-13T23:43:25.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f000 2024-09-13T23:43:25.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.379+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c000 2024-09-13T23:43:25.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f000 2024-09-13T23:43:25.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f400 2024-09-13T23:43:25.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f400 2024-09-13T23:43:25.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f000 2024-09-13T23:43:25.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c400 2024-09-13T23:43:25.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f000 2024-09-13T23:43:25.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.383+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c400 2024-09-13T23:43:25.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.387+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.387+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f400 2024-09-13T23:43:25.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.387+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.387+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c000 2024-09-13T23:43:25.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.387+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.387+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f000 2024-09-13T23:43:25.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.387+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.387+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f400 2024-09-13T23:43:25.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.387+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.387+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c000 2024-09-13T23:43:25.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.387+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.387+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f000 2024-09-13T23:43:25.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f000 2024-09-13T23:43:25.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f000 2024-09-13T23:43:25.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c000 2024-09-13T23:43:25.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c400 2024-09-13T23:43:25.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f000 2024-09-13T23:43:25.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c400 2024-09-13T23:43:25.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.391+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c000 2024-09-13T23:43:25.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.395+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.395+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c400 2024-09-13T23:43:25.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.395+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.395+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f400 2024-09-13T23:43:25.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.395+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.395+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f400 2024-09-13T23:43:25.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.395+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.395+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c000 2024-09-13T23:43:25.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.395+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.395+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f400 2024-09-13T23:43:25.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623baf6f400 2024-09-13T23:43:25.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:25.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:25.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:25.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:25.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01c800 con 0x5623baf6f000 2024-09-13T23:43:25.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01cc00 con 0x5623baf6f400 2024-09-13T23:43:25.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:25.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d000 con 0x5623baf6f400 2024-09-13T23:43:25.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:25.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 zone default found 2024-09-13T23:43:25.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 4 Realm: () 2024-09-13T23:43:25.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:25.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:25.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:25.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 zonegroup default 2024-09-13T23:43:25.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d400 con 0x5623bb01c400 2024-09-13T23:43:25.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.399+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d800 con 0x5623bb01c000 2024-09-13T23:43:25.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.403+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.403+0000 7f84fa86cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.403+0000 7f84fa86cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.403+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01dc00 con 0x5623bb01c000 2024-09-13T23:43:25.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.403+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.403+0000 7f84fa86cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.403+0000 7f84fa86cf40 20 started sync module instance, tier type = 2024-09-13T23:43:25.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.403+0000 7f84fa86cf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:25.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.403+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d800 con 0x5623baf6f400 2024-09-13T23:43:25.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.403+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.407+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94711459699456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01dc00 con 0x5623baf6f400 2024-09-13T23:43:25.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.407+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01cc00 con 0x5623baf6f000 2024-09-13T23:43:25.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.407+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.407+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94711459700608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d000 con 0x5623baf6f000 2024-09-13T23:43:25.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.407+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d800 con 0x5623baf6f400 2024-09-13T23:43:25.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.407+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94711459699456] v33'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.411+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94711459700608] v33'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.411+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.411+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94711459701760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6fc00 con 0x5623baf6f400 2024-09-13T23:43:25.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.411+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6f800 con 0x5623bb01c400 2024-09-13T23:43:25.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.415+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94711459701760] v33'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.415+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.415+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94711459702912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01c800 con 0x5623bb01c400 2024-09-13T23:43:25.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.415+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d400 con 0x5623baf6f400 2024-09-13T23:43:25.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.415+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94711459702912] v33'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.415+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.415+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94711459704064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01cc00 con 0x5623baf6f400 2024-09-13T23:43:25.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.415+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d000 con 0x5623bb01c000 2024-09-13T23:43:25.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.419+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.419+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94711459705216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02a000 con 0x5623bb01c000 2024-09-13T23:43:25.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.419+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02a400 con 0x5623bb01c000 2024-09-13T23:43:25.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.419+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94711459704064] v33'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.419+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94711459705216] v33'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.419+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.419+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94711459706368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02a800 con 0x5623bb01c000 2024-09-13T23:43:25.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.419+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02ac00 con 0x5623bb01c400 2024-09-13T23:43:25.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'15 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94711459707520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02b000 con 0x5623bb01c400 2024-09-13T23:43:25.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84fa86cf40 20 add_watcher() i=0 2024-09-13T23:43:25.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84fa86cf40 20 add_watcher() i=1 2024-09-13T23:43:25.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84fa86cf40 20 add_watcher() i=2 2024-09-13T23:43:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84fa86cf40 20 add_watcher() i=3 2024-09-13T23:43:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84fa86cf40 20 add_watcher() i=4 2024-09-13T23:43:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84fa86cf40 20 add_watcher() i=5 2024-09-13T23:43:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94711459706368] v33'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84fa86cf40 20 add_watcher() i=6 2024-09-13T23:43:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94711459707520] v33'16 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84fa86cf40 20 add_watcher() i=7 2024-09-13T23:43:25.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84fa86cf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:25.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.423+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02b400 con 0x5623baf6f000 2024-09-13T23:43:25.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94711461380096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02b800 con 0x5623baf6f000 2024-09-13T23:43:25.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94711461380096] v33'41 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84fa86cf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:25.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84fa86cf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:25.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84fa86cf40 1 Processor -- start 2024-09-13T23:43:25.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84dfff3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:25.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84fa86cf40 1 -- start start 2024-09-13T23:43:25.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84fa86cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 0x5623baea1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84fa86cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bac463c0 con 0x5623bb02ac00 2024-09-13T23:43:25.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84f8824700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 0x5623baea1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84f8824700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 0x5623baea1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43168/0 (socket says 172.21.15.167:43168) 2024-09-13T23:43:25.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.427+0000 7f84f8824700 1 -- 172.21.15.167:0/2041578580 learned_addr learned my addr 172.21.15.167:0/2041578580 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84f8824700 1 -- 172.21.15.167:0/2041578580 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623babcab60 con 0x5623bb02ac00 2024-09-13T23:43:25.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84f8824700 1 --2- 172.21.15.167:0/2041578580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 0x5623baea1180 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5623bba2a180 tx=0x5623bacbd860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e194a252440ffde server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84df7f2700 1 -- 172.21.15.167:0/2041578580 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bac5d760 con 0x5623bb02ac00 2024-09-13T23:43:25.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84df7f2700 1 -- 172.21.15.167:0/2041578580 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623baf12600 con 0x5623bb02ac00 2024-09-13T23:43:25.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84df7f2700 1 -- 172.21.15.167:0/2041578580 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bb044000 con 0x5623bb02ac00 2024-09-13T23:43:25.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2041578580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 msgr2=0x5623baea1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2041578580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 0x5623baea1180 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5623bba2a180 tx=0x5623bacbd860 comp rx=0 tx=0).stop 2024-09-13T23:43:25.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2041578580 shutdown_connections 2024-09-13T23:43:25.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2041578580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 0x5623baea1180 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2041578580 >> 172.21.15.167:0/2041578580 conn(0x5623bb02bc00 msgr2=0x5623bacba000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2041578580 shutdown_connections 2024-09-13T23:43:25.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2041578580 wait complete. 2024-09-13T23:43:25.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84fa86cf40 1 Processor -- start 2024-09-13T23:43:25.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84fa86cf40 1 -- start start 2024-09-13T23:43:25.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84fa86cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 0x5623baea1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84fa86cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5623bac46500 con 0x5623bb02ac00 2024-09-13T23:43:25.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84f8824700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 0x5623baea1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84f8824700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 0x5623baea1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43170/0 (socket says 172.21.15.167:43170) 2024-09-13T23:43:25.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84f8824700 1 -- 172.21.15.167:0/2515401709 learned_addr learned my addr 172.21.15.167:0/2515401709 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.431+0000 7f84f8824700 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5623babcad00 con 0x5623bb02ac00 2024-09-13T23:43:25.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84f8824700 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 0x5623baea1180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x5623bb0387b0 tx=0x5623bb03a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84ddfef700 1 -- 172.21.15.167:0/2515401709 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bb044160 con 0x5623bb02ac00 2024-09-13T23:43:25.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84ddfef700 1 -- 172.21.15.167:0/2515401709 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5623baf12300 con 0x5623bb02ac00 2024-09-13T23:43:25.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5623babca340 con 0x5623bb02ac00 2024-09-13T23:43:25.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84ddfef700 1 -- 172.21.15.167:0/2515401709 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5623bb0442c0 con 0x5623bb02ac00 2024-09-13T23:43:25.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5623babcbd40 con 0x5623bb02ac00 2024-09-13T23:43:25.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84ddfef700 1 -- 172.21.15.167:0/2515401709 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5623bacb3b80 con 0x5623bb02ac00 2024-09-13T23:43:25.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84ddfef700 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623bb0b8000 0x5623bb0be000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84ddfef700 1 -- 172.21.15.167:0/2515401709 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x5623bac1f500 con 0x5623bb02ac00 2024-09-13T23:43:25.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84f9826700 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623bb0b8000 0x5623bb0be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5623bb02b000 0x5623baea1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02b400 con 0x5623bb02b000 2024-09-13T23:43:25.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84f9025700 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5623bb02b000 0x5623baea1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84f9826700 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623bb0b8000 0x5623bb0be000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5623bafb3cb0 tx=0x5623baf217a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.435+0000 7f84f9025700 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5623bb02b000 0x5623baea1700 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.439+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.439+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5623bb02a000 0x5623bb0be580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.439+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02a400 con 0x5623bb02a000 2024-09-13T23:43:25.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.439+0000 7f84f8824700 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5623bb02a000 0x5623bb0be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.439+0000 7f84f8824700 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5623bb02a000 0x5623bb0be580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.443+0000 7f84f8824700 1 -- 172.21.15.167:0/2515401709 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'13 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb02a000 2024-09-13T23:43:25.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.443+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5623bb01c800 0x5623bb0beb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.443+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d000 con 0x5623bb01c800 2024-09-13T23:43:25.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.443+0000 7f84f9826700 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5623bb01c800 0x5623bb0beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.443+0000 7f84f9826700 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5623bb01c800 0x5623bb0beb00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.443+0000 7f84f9826700 1 -- 172.21.15.167:0/2515401709 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'13 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c800 2024-09-13T23:43:25.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.443+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5623baf6f800 0x5623bb0bf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.443+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b8400 con 0x5623baf6f800 2024-09-13T23:43:25.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.443+0000 7f84f9025700 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5623baf6f800 0x5623bb0bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.443+0000 7f84f9025700 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5623baf6f800 0x5623bb0bf080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.447+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'42 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f800 2024-09-13T23:43:25.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.447+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b8800 con 0x5623bb02b000 2024-09-13T23:43:25.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.451+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'29 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.451+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b8c00 con 0x5623bb01c800 2024-09-13T23:43:25.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.451+0000 7f84f9826700 1 -- 172.21.15.167:0/2515401709 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'19 uv10 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c800 2024-09-13T23:43:25.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.451+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b9000 con 0x5623bb02b000 2024-09-13T23:43:25.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.455+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'15 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.455+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b9400 con 0x5623baf6f800 2024-09-13T23:43:25.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.459+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'19 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f800 2024-09-13T23:43:25.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.459+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b9800 con 0x5623bb02b000 2024-09-13T23:43:25.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.459+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.459+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b9c00 con 0x5623bb02b000 2024-09-13T23:43:25.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.463+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'19 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.463+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02b400 con 0x5623bb02b000 2024-09-13T23:43:25.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.467+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'30 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.467+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01dc00 con 0x5623bb01c800 2024-09-13T23:43:25.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.467+0000 7f84f9826700 1 -- 172.21.15.167:0/2515401709 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'15 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c800 2024-09-13T23:43:25.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.467+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d800 con 0x5623bb02b000 2024-09-13T23:43:25.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.471+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.471+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d400 con 0x5623baf6f800 2024-09-13T23:43:25.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.475+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'25 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f800 2024-09-13T23:43:25.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.475+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01cc00 con 0x5623baf6f800 2024-09-13T23:43:25.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.475+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'43 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f800 2024-09-13T23:43:25.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.475+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6fc00 con 0x5623bb02b000 2024-09-13T23:43:25.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.479+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.479+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02b400 con 0x5623bb02b000 2024-09-13T23:43:25.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.483+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'31 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.483+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b8400 con 0x5623bb01c800 2024-09-13T23:43:25.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.483+0000 7f84f9826700 1 -- 172.21.15.167:0/2515401709 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'13 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c800 2024-09-13T23:43:25.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.483+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b8800 con 0x5623bb02b000 2024-09-13T23:43:25.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.487+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'21 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.487+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b9000 con 0x5623baf6f800 2024-09-13T23:43:25.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.487+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'44 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f800 2024-09-13T23:43:25.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.491+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b9400 con 0x5623bb02b000 2024-09-13T23:43:25.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.491+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'24 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.491+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b9800 con 0x5623bb02b000 2024-09-13T23:43:25.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.495+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'23 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.495+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b9c00 con 0x5623bb01c800 2024-09-13T23:43:25.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.495+0000 7f84f9826700 1 -- 172.21.15.167:0/2515401709 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c800 2024-09-13T23:43:25.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.499+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b8c00 con 0x5623bb02b000 2024-09-13T23:43:25.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.499+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.499+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b8400 con 0x5623bb01c800 2024-09-13T23:43:25.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.503+0000 7f84f9826700 1 -- 172.21.15.167:0/2515401709 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'9 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c800 2024-09-13T23:43:25.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.503+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b9c00 con 0x5623bb01c800 2024-09-13T23:43:25.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.507+0000 7f84f9826700 1 -- 172.21.15.167:0/2515401709 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'20 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c800 2024-09-13T23:43:25.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.507+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b8800 con 0x5623bb01c800 2024-09-13T23:43:25.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.507+0000 7f84f9826700 1 -- 172.21.15.167:0/2515401709 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c800 2024-09-13T23:43:25.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.507+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b9000 con 0x5623bb02b000 2024-09-13T23:43:25.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.511+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'13 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.511+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb0b9400 con 0x5623baf6f800 2024-09-13T23:43:25.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.515+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'20 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f800 2024-09-13T23:43:25.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.515+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02b800 con 0x5623bb02b000 2024-09-13T23:43:25.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.515+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'25 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623bb02b000 2024-09-13T23:43:25.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.515+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb02b400 con 0x5623baf6f800 2024-09-13T23:43:25.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.519+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'26 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f800 2024-09-13T23:43:25.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.519+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf6fc00 con 0x5623bb02a000 2024-09-13T23:43:25.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.519+0000 7f84f8824700 1 -- 172.21.15.167:0/2515401709 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb02a000 2024-09-13T23:43:25.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.523+0000 7f84fa86cf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:25.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.523+0000 7f84fa86cf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:25.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.523+0000 7f84d97e6700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:55 2024 2024-09-13T23:43:25.533 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:25.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.523+0000 7f84d8fc4700 20 reqs_thread_entry: start 2024-09-13T23:43:25.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.523+0000 7f84fa86cf40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:25.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.523+0000 7f84fa86cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.523+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d800 con 0x5623baf6f000 2024-09-13T23:43:25.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.523+0000 7f84db7ea700 20 reqs_thread_entry: start 2024-09-13T23:43:25.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.523+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.523+0000 7f84fa86cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.523+0000 7f84fa86cf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-09-13T23:43:25.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.523+0000 7f84fa86cf40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:25.534 INFO:teuthology.orchestra.run.smithi167.stderr:could not remove user: unable to remove user, user does not exist 2024-09-13T23:43:25.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.527+0000 7f84dfff3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:25.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.527+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94711461380096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01d400 con 0x5623baf6f000 2024-09-13T23:43:25.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.527+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94711461380096] v33'45 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.527+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94711459699456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623bb01cc00 con 0x5623baf6f400 2024-09-13T23:43:25.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.531+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94711459699456] v33'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.531+0000 7f84fa86cf40 20 remove_watcher() i=0 2024-09-13T23:43:25.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.531+0000 7f84fa86cf40 2 removed watcher, disabling cache 2024-09-13T23:43:25.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.531+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94711459700608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf30000 con 0x5623baf6f000 2024-09-13T23:43:25.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.535+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94711459700608] v33'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623baf6f000 2024-09-13T23:43:25.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.535+0000 7f84fa86cf40 20 remove_watcher() i=1 2024-09-13T23:43:25.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.535+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94711459701760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf30400 con 0x5623baf6f400 2024-09-13T23:43:25.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.535+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94711459701760] v33'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.535+0000 7f84fa86cf40 20 remove_watcher() i=2 2024-09-13T23:43:25.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.535+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94711459702912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf30800 con 0x5623bb01c400 2024-09-13T23:43:25.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.539+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94711459702912] v33'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.539+0000 7f84fa86cf40 20 remove_watcher() i=3 2024-09-13T23:43:25.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.539+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94711459704064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf30c00 con 0x5623baf6f400 2024-09-13T23:43:25.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.543+0000 7f84f9025700 1 -- 172.21.15.167:0/349458305 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94711459704064] v33'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b200 con 0x5623baf6f400 2024-09-13T23:43:25.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.543+0000 7f84fa86cf40 20 remove_watcher() i=4 2024-09-13T23:43:25.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.543+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94711459705216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf31000 con 0x5623bb01c000 2024-09-13T23:43:25.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.543+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94711459705216] v33'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.543+0000 7f84fa86cf40 20 remove_watcher() i=5 2024-09-13T23:43:25.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.543+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94711459706368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf31400 con 0x5623bb01c000 2024-09-13T23:43:25.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.547+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94711459706368] v33'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00b440 con 0x5623bb01c000 2024-09-13T23:43:25.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.547+0000 7f84fa86cf40 20 remove_watcher() i=6 2024-09-13T23:43:25.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.547+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94711459707520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5623baf31800 con 0x5623bb01c400 2024-09-13T23:43:25.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.547+0000 7f84f9826700 1 -- 172.21.15.167:0/349458305 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94711459707520] v33'17 uv15 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5623bb00ab40 con 0x5623bb01c400 2024-09-13T23:43:25.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.547+0000 7f84fa86cf40 20 remove_watcher() i=7 2024-09-13T23:43:25.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5623baf6f400 msgr2=0x5623baea0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5623baf6f400 0x5623baea0100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5623baf6f000 msgr2=0x5623bae9fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5623baf6f000 0x5623bae9fb80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5623bb01c400 msgr2=0x5623baea0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5623bb01c400 0x5623baea0c00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5623bb01c000 msgr2=0x5623baea0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5623bb01c000 0x5623baea0680 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623baf6ec00 msgr2=0x5623bae9f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623baf6ec00 0x5623bae9f600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x5623bae99920 tx=0x5623bacbd800 comp rx=0 tx=0).stop 2024-09-13T23:43:25.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 msgr2=0x5623bae9eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84f8824700 1 -- 172.21.15.167:0/349458305 reap_dead start 2024-09-13T23:43:25.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 0x5623bae9eb00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5623bba2a3f0 tx=0x5623baf202a0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 shutdown_connections 2024-09-13T23:43:25.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/349458305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9800 0x5623bae9eb00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 >> 172.21.15.167:0/349458305 conn(0x5623babb9400 msgr2=0x5623bacb9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 shutdown_connections 2024-09-13T23:43:25.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/349458305 wait complete. 2024-09-13T23:43:25.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5623bb02b000 msgr2=0x5623baea1700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5623bb02b000 0x5623baea1700 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5623baf6f800 msgr2=0x5623bb0bf080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5623baf6f800 0x5623bb0bf080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5623bb02a000 msgr2=0x5623bb0be580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5623bb02a000 0x5623bb0be580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5623bb01c800 msgr2=0x5623bb0beb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5623bb01c800 0x5623bb0beb00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623bb0b8000 msgr2=0x5623bb0be000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623bb0b8000 0x5623bb0be000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x5623bafb3cb0 tx=0x5623baf217a0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 msgr2=0x5623baea1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84f9025700 1 -- 172.21.15.167:0/2515401709 reap_dead start 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 0x5623baea1180 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x5623bb0387b0 tx=0x5623bb03a300 comp rx=0 tx=0).stop 2024-09-13T23:43:25.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 shutdown_connections 2024-09-13T23:43:25.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/2515401709 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623bb02ac00 0x5623baea1180 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 >> 172.21.15.167:0/2515401709 conn(0x5623bb02bc00 msgr2=0x5623bacba000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 shutdown_connections 2024-09-13T23:43:25.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/2515401709 wait complete. 2024-09-13T23:43:25.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/3834612799 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623baf6e000 msgr2=0x5623bae9f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/3834612799 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623baf6e000 0x5623bae9f080 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x5623baf4a660 tx=0x5623baf20240 comp rx=0 tx=0).stop 2024-09-13T23:43:25.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/3834612799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 msgr2=0x5623bae9e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/3834612799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 0x5623bae9e580 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x5623bba2a0f0 tx=0x5623bacbda40 comp rx=0 tx=0).stop 2024-09-13T23:43:25.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/3834612799 shutdown_connections 2024-09-13T23:43:25.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/3834612799 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5623baf6e000 0x5623bae9f080 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 --2- 172.21.15.167:0/3834612799 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5623babb9000 0x5623bae9e580 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/3834612799 >> 172.21.15.167:0/3834612799 conn(0x5623babb8c00 msgr2=0x5623bacb9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/3834612799 shutdown_connections 2024-09-13T23:43:25.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.551+0000 7f84fa86cf40 1 -- 172.21.15.167:0/3834612799 wait complete. 2024-09-13T23:43:25.567 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-13T23:43:25.567 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-09-13T23:43:25.567 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-09-13T23:43:25.568 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-09-13T23:43:25.653 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:25.654 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:25.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 Processor -- start 2024-09-13T23:43:25.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 -- start start 2024-09-13T23:43:25.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55cb052d7400 0x55cb055bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55cb0539b500 con 0x55cb052d7000 2024-09-13T23:43:25.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55cb05364a00 con 0x55cb052d7400 2024-09-13T23:43:25.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a3ef0b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55cb052d7400 0x55cb055bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a3ef0b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55cb052d7400 0x55cb055bc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43182/0 (socket says 172.21.15.167:43182) 2024-09-13T23:43:25.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a3ef0b700 1 -- 172.21.15.167:0/2275514348 learned_addr learned my addr 172.21.15.167:0/2275514348 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a3f70c700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55cb052d7000 0x55cb053d6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:40338/0 (socket says 172.21.15.167:40338) 2024-09-13T23:43:25.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a3ef0b700 1 -- 172.21.15.167:0/2275514348 >> v1:172.21.15.12:6789/0 conn(0x55cb052d7000 legacy=0x55cb053d6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a3ef0b700 1 -- 172.21.15.167:0/2275514348 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb052e9d40 con 0x55cb052d7400 2024-09-13T23:43:25.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a3ef0b700 1 --2- 172.21.15.167:0/2275514348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7400 0x55cb055bc580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55cb055b7920 tx=0x55cb053db920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861474bab9a7827c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a3e70a700 1 -- 172.21.15.167:0/2275514348 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537b4a0 con 0x55cb052d7400 2024-09-13T23:43:25.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a3e70a700 1 -- 172.21.15.167:0/2275514348 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb0539b800 con 0x55cb052d7400 2024-09-13T23:43:25.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a3e70a700 1 -- 172.21.15.167:0/2275514348 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537b600 con 0x55cb052d7400 2024-09-13T23:43:25.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2275514348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7400 msgr2=0x55cb055bc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/2275514348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7400 0x55cb055bc580 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55cb055b7920 tx=0x55cb053db920 comp rx=0 tx=0).stop 2024-09-13T23:43:25.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2275514348 shutdown_connections 2024-09-13T23:43:25.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/2275514348 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7400 0x55cb055bc580 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2275514348 >> 172.21.15.167:0/2275514348 conn(0x55cb052d6c00 msgr2=0x55cb053d6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2275514348 shutdown_connections 2024-09-13T23:43:25.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2275514348 wait complete. 2024-09-13T23:43:25.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 Processor -- start 2024-09-13T23:43:25.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.647+0000 7f7a40f53f40 1 -- start start 2024-09-13T23:43:25.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 0x55cb055bc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb05364500 con 0x55cb052d7000 2024-09-13T23:43:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a3f70c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 0x55cb055bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a3f70c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 0x55cb055bc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43190/0 (socket says 172.21.15.167:43190) 2024-09-13T23:43:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a3f70c700 1 -- 172.21.15.167:0/1601767539 learned_addr learned my addr 172.21.15.167:0/1601767539 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a3f70c700 1 -- 172.21.15.167:0/1601767539 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb052e9ba0 con 0x55cb052d7000 2024-09-13T23:43:25.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a3f70c700 1 --2- 172.21.15.167:0/1601767539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 0x55cb055bc580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55cb05433e90 tx=0x55cb053dbb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a486e46115de5cfe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a3df09700 1 -- 172.21.15.167:0/1601767539 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537b4a0 con 0x55cb052d7000 2024-09-13T23:43:25.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a3df09700 1 -- 172.21.15.167:0/1601767539 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb0539bb00 con 0x55cb052d7000 2024-09-13T23:43:25.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a3df09700 1 -- 172.21.15.167:0/1601767539 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537b600 con 0x55cb052d7000 2024-09-13T23:43:25.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1601767539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 msgr2=0x55cb055bc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1601767539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 0x55cb055bc580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55cb05433e90 tx=0x55cb053dbb60 comp rx=0 tx=0).stop 2024-09-13T23:43:25.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1601767539 shutdown_connections 2024-09-13T23:43:25.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1601767539 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 0x55cb055bc580 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1601767539 >> 172.21.15.167:0/1601767539 conn(0x55cb052d6c00 msgr2=0x55cb053d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1601767539 shutdown_connections 2024-09-13T23:43:25.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1601767539 wait complete. 2024-09-13T23:43:25.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 Processor -- start 2024-09-13T23:43:25.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 -- start start 2024-09-13T23:43:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 0x55cb055bc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a40f53f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb053643c0 con 0x55cb052d7000 2024-09-13T23:43:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a3f70c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 0x55cb055bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a3f70c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 0x55cb055bc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43204/0 (socket says 172.21.15.167:43204) 2024-09-13T23:43:25.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.651+0000 7f7a3f70c700 1 -- 172.21.15.167:0/1785819985 learned_addr learned my addr 172.21.15.167:0/1785819985 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3f70c700 1 -- 172.21.15.167:0/1785819985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb052e9a00 con 0x55cb052d7000 2024-09-13T23:43:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3f70c700 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 0x55cb055bc580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55cb05433fb0 tx=0x55cb053dba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3c706700 1 -- 172.21.15.167:0/1785819985 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537b8c0 con 0x55cb052d7000 2024-09-13T23:43:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1785819985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cb052e9860 con 0x55cb052d7000 2024-09-13T23:43:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3c706700 1 -- 172.21.15.167:0/1785819985 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb0539bc80 con 0x55cb052d7000 2024-09-13T23:43:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3c706700 1 -- 172.21.15.167:0/1785819985 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537b4a0 con 0x55cb052d7000 2024-09-13T23:43:25.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1785819985 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb052e96c0 con 0x55cb052d7000 2024-09-13T23:43:25.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a40f53f40 1 Processor -- start 2024-09-13T23:43:25.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a40f53f40 1 -- start start 2024-09-13T23:43:25.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3c706700 1 -- 172.21.15.167:0/1785819985 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55cb053d0b00 con 0x55cb052d7000 2024-09-13T23:43:25.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a40f53f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 0x55cb055bcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a40f53f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb05364000 con 0x55cb052d7800 2024-09-13T23:43:25.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3ff0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 0x55cb055bcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3ff0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 0x55cb055bcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43210/0 (socket says 172.21.15.167:43210) 2024-09-13T23:43:25.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3ff0d700 1 -- 172.21.15.167:0/1796575937 learned_addr learned my addr 172.21.15.167:0/1796575937 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3c706700 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb05680000 0x55cb055bd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3c706700 1 -- 172.21.15.167:0/1785819985 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55cb0533d180 con 0x55cb052d7000 2024-09-13T23:43:25.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3f70c700 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb05680000 0x55cb055bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3ff0d700 1 -- 172.21.15.167:0/1796575937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb052e8000 con 0x55cb052d7800 2024-09-13T23:43:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.655+0000 7f7a3ff0d700 1 --2- 172.21.15.167:0/1796575937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 0x55cb055bcb00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55cb0565e510 tx=0x55cb053dbf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34298c6056dc03b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a3f70c700 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb05680000 0x55cb055bd080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55cb0565e570 tx=0x55cb05636240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a39f01700 1 -- 172.21.15.167:0/1796575937 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537b8c0 con 0x55cb052d7800 2024-09-13T23:43:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a39f01700 1 -- 172.21.15.167:0/1796575937 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb0569a000 con 0x55cb052d7800 2024-09-13T23:43:25.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a39f01700 1 -- 172.21.15.167:0/1796575937 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537b4a0 con 0x55cb052d7800 2024-09-13T23:43:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1796575937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 msgr2=0x55cb055bcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1796575937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 0x55cb055bcb00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x55cb0565e510 tx=0x55cb053dbf20 comp rx=0 tx=0).stop 2024-09-13T23:43:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1796575937 shutdown_connections 2024-09-13T23:43:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1796575937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 0x55cb055bcb00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1796575937 >> 172.21.15.167:0/1796575937 conn(0x55cb052d7400 msgr2=0x55cb053d7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1796575937 shutdown_connections 2024-09-13T23:43:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1796575937 wait complete. 2024-09-13T23:43:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 Processor -- start 2024-09-13T23:43:25.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 -- start start 2024-09-13T23:43:25.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 0x55cb055bcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb052fc640 con 0x55cb052d7800 2024-09-13T23:43:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a3ff0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 0x55cb055bcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a3ff0d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 0x55cb055bcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43226/0 (socket says 172.21.15.167:43226) 2024-09-13T23:43:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a3ff0d700 1 -- 172.21.15.167:0/3082694454 learned_addr learned my addr 172.21.15.167:0/3082694454 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a3ff0d700 1 -- 172.21.15.167:0/3082694454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb052e8ea0 con 0x55cb052d7800 2024-09-13T23:43:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a3ff0d700 1 --2- 172.21.15.167:0/3082694454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 0x55cb055bcb00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55cb0614e930 tx=0x55cb053dbe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a386fe700 1 -- 172.21.15.167:0/3082694454 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537bb80 con 0x55cb052d7800 2024-09-13T23:43:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a386fe700 1 -- 172.21.15.167:0/3082694454 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb0569a180 con 0x55cb052d7800 2024-09-13T23:43:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 -- 172.21.15.167:0/3082694454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cb052e8820 con 0x55cb052d7800 2024-09-13T23:43:25.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a386fe700 1 -- 172.21.15.167:0/3082694454 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537b8c0 con 0x55cb052d7800 2024-09-13T23:43:25.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.659+0000 7f7a40f53f40 1 -- 172.21.15.167:0/3082694454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb052e89c0 con 0x55cb052d7800 2024-09-13T23:43:25.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a386fe700 1 -- 172.21.15.167:0/3082694454 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55cb053d0dc0 con 0x55cb052d7800 2024-09-13T23:43:25.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a40f53f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:25.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a40f53f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:25.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a40f53f40 1 Processor -- start 2024-09-13T23:43:25.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a40f53f40 1 -- start start 2024-09-13T23:43:25.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a386fe700 1 --2- 172.21.15.167:0/3082694454 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb05681c00 0x55cb055bd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a386fe700 1 -- 172.21.15.167:0/3082694454 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55cb0533d340 con 0x55cb052d7800 2024-09-13T23:43:25.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a40f53f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 0x55cb055bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a3ff0d700 1 --2- 172.21.15.167:0/3082694454 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb05681c00 0x55cb055bd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a40f53f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb052fc8c0 con 0x55cb05681000 2024-09-13T23:43:25.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a3f70c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 0x55cb055bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a3f70c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 0x55cb055bdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43242/0 (socket says 172.21.15.167:43242) 2024-09-13T23:43:25.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a3f70c700 1 -- 172.21.15.167:0/1738242433 learned_addr learned my addr 172.21.15.167:0/1738242433 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a3ff0d700 1 --2- 172.21.15.167:0/3082694454 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb05681c00 0x55cb055bd600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55cb056d8960 tx=0x55cb056377a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a3f70c700 1 -- 172.21.15.167:0/1738242433 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb052e8b60 con 0x55cb05681000 2024-09-13T23:43:25.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a3f70c700 1 --2- 172.21.15.167:0/1738242433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 0x55cb055bdb80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55cb0565e6f0 tx=0x55cb056370e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcde87b177c6e353 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.663+0000 7f7a266da700 1 -- 172.21.15.167:0/1738242433 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537b600 con 0x55cb05681000 2024-09-13T23:43:25.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a266da700 1 -- 172.21.15.167:0/1738242433 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb0569a600 con 0x55cb05681000 2024-09-13T23:43:25.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a266da700 1 -- 172.21.15.167:0/1738242433 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537ba20 con 0x55cb05681000 2024-09-13T23:43:25.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1738242433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 msgr2=0x55cb055bdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1738242433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 0x55cb055bdb80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55cb0565e6f0 tx=0x55cb056370e0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1738242433 shutdown_connections 2024-09-13T23:43:25.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1738242433 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 0x55cb055bdb80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1738242433 >> 172.21.15.167:0/1738242433 conn(0x55cb05680c00 msgr2=0x55cb053d8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1738242433 shutdown_connections 2024-09-13T23:43:25.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1738242433 wait complete. 2024-09-13T23:43:25.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 Processor -- start 2024-09-13T23:43:25.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 -- start start 2024-09-13T23:43:25.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 0x55cb055bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb052fcf00 con 0x55cb05681000 2024-09-13T23:43:25.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a3f70c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 0x55cb055bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a3f70c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 0x55cb055bdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43246/0 (socket says 172.21.15.167:43246) 2024-09-13T23:43:25.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a3f70c700 1 -- 172.21.15.167:0/2279701968 learned_addr learned my addr 172.21.15.167:0/2279701968 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a3f70c700 1 -- 172.21.15.167:0/2279701968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb052e8d00 con 0x55cb05681000 2024-09-13T23:43:25.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a3f70c700 1 --2- 172.21.15.167:0/2279701968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 0x55cb055bdb80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55cb056d9da0 tx=0x55cb056361e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a24ed7700 1 -- 172.21.15.167:0/2279701968 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537b600 con 0x55cb05681000 2024-09-13T23:43:25.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a24ed7700 1 -- 172.21.15.167:0/2279701968 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb0569a300 con 0x55cb05681000 2024-09-13T23:43:25.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2279701968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cb052e8340 con 0x55cb05681000 2024-09-13T23:43:25.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a24ed7700 1 -- 172.21.15.167:0/2279701968 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cb0537ba20 con 0x55cb05681000 2024-09-13T23:43:25.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.667+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2279701968 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb05746000 con 0x55cb05681000 2024-09-13T23:43:25.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.671+0000 7f7a24ed7700 1 -- 172.21.15.167:0/2279701968 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55cb053d1b80 con 0x55cb05681000 2024-09-13T23:43:25.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.671+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cb0573c800 0x55cb055be100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.671+0000 7f7a24ed7700 1 --2- 172.21.15.167:0/2279701968 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb0573dc00 0x55cb055bec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.671+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1785819985 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cb0573cc00 con 0x55cb0573c800 2024-09-13T23:43:25.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.671+0000 7f7a24ed7700 1 -- 172.21.15.167:0/2279701968 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55cb0533d500 con 0x55cb05681000 2024-09-13T23:43:25.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.671+0000 7f7a22ed3700 20 reqs_thread_entry: start 2024-09-13T23:43:25.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.671+0000 7f7a3ff0d700 1 --2- 172.21.15.167:0/2279701968 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb0573dc00 0x55cb055bec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.671+0000 7f7a3ef0b700 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cb0573c800 0x55cb055be100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.671+0000 7f7a3ff0d700 1 --2- 172.21.15.167:0/2279701968 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb0573dc00 0x55cb055bec00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55cb056d8a20 tx=0x55cb05744f60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.671+0000 7f7a3ef0b700 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cb0573c800 0x55cb055be100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.671+0000 7f7a3ef0b700 1 -- 172.21.15.167:0/1785819985 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55cb057b4240 con 0x55cb0573c800 2024-09-13T23:43:25.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.671+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1785819985 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:2 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cb0573d000 con 0x55cb0573c800 2024-09-13T23:43:25.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.675+0000 7f7a3ef0b700 1 -- 172.21.15.167:0/1785819985 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(2 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [call out=24b,read 0~874 out=874b] v0'0 uv1 ondisk = 0) v8 ==== 232+0+898 (crc 0 0 0) 0x55cb057b4240 con 0x55cb0573c800 2024-09-13T23:43:25.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.675+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1785819985 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:3 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [stat,call version.inc in=38b,writefull 0~874 in=874b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cb0573d400 con 0x55cb0573c800 2024-09-13T23:43:25.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a3ef0b700 1 -- 172.21.15.167:0/1785819985 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(3 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [stat,call,writefull 0~874] v33'2 uv2 ondisk = 0) v8 ==== 274+0+0 (crc 0 0 0) 0x55cb057b4240 con 0x55cb0573c800 2024-09-13T23:43:25.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/3082694454 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb05681c00 msgr2=0x55cb055bd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/3082694454 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb05681c00 0x55cb055bd600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55cb056d8960 tx=0x55cb056377a0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/3082694454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 msgr2=0x55cb055bcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/3082694454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 0x55cb055bcb00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x55cb0614e930 tx=0x55cb053dbe00 comp rx=0 tx=0).stop 2024-09-13T23:43:25.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/3082694454 shutdown_connections 2024-09-13T23:43:25.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/3082694454 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb05681c00 0x55cb055bd600 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/3082694454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7800 0x55cb055bcb00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/3082694454 >> 172.21.15.167:0/3082694454 conn(0x55cb052d7400 msgr2=0x55cb053d7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/3082694454 shutdown_connections 2024-09-13T23:43:25.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/3082694454 wait complete. 2024-09-13T23:43:25.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2279701968 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb0573dc00 msgr2=0x55cb055bec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/2279701968 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb0573dc00 0x55cb055bec00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x55cb056d8a20 tx=0x55cb05744f60 comp rx=0 tx=0).stop 2024-09-13T23:43:25.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2279701968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 msgr2=0x55cb055bdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/2279701968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 0x55cb055bdb80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55cb056d9da0 tx=0x55cb056361e0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2279701968 shutdown_connections 2024-09-13T23:43:25.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/2279701968 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb0573dc00 0x55cb055bec00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/2279701968 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb05681000 0x55cb055bdb80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2279701968 >> 172.21.15.167:0/2279701968 conn(0x55cb05680c00 msgr2=0x55cb053d8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2279701968 shutdown_connections 2024-09-13T23:43:25.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/2279701968 wait complete. 2024-09-13T23:43:25.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1785819985 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cb0573c800 msgr2=0x55cb055be100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cb0573c800 0x55cb055be100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1785819985 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb05680000 msgr2=0x55cb055bd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.679+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb05680000 0x55cb055bd080 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55cb0565e570 tx=0x55cb05636240 comp rx=0 tx=0).stop 2024-09-13T23:43:25.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.683+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1785819985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 msgr2=0x55cb055bc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.683+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 0x55cb055bc580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55cb05433fb0 tx=0x55cb053dba40 comp rx=0 tx=0).stop 2024-09-13T23:43:25.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.683+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1785819985 shutdown_connections 2024-09-13T23:43:25.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.683+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cb0573c800 0x55cb055be100 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.683+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cb05680000 0x55cb055bd080 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.683+0000 7f7a40f53f40 1 --2- 172.21.15.167:0/1785819985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cb052d7000 0x55cb055bc580 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.683+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1785819985 >> 172.21.15.167:0/1785819985 conn(0x55cb052d6c00 msgr2=0x55cb053d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.683+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1785819985 shutdown_connections 2024-09-13T23:43:25.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.683+0000 7f7a40f53f40 1 -- 172.21.15.167:0/1785819985 wait complete. 2024-09-13T23:43:25.693 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:25.693 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "9144208c-3c85-4fb6-9007-2693d5a930f0", 2024-09-13T23:43:25.693 INFO:teuthology.orchestra.run.smithi167.stdout: "name": "default", 2024-09-13T23:43:25.693 INFO:teuthology.orchestra.run.smithi167.stdout: "domain_root": "default.rgw.meta:root", 2024-09-13T23:43:25.693 INFO:teuthology.orchestra.run.smithi167.stdout: "control_pool": "default.rgw.control", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "gc_pool": "default.rgw.log:gc", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "lc_pool": "default.rgw.log:lc", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "log_pool": "default.rgw.log", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "roles_pool": "default.rgw.meta:roles", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "otp_pool": "default.rgw.otp", 2024-09-13T23:43:25.694 INFO:teuthology.orchestra.run.smithi167.stdout: "system_key": { 2024-09-13T23:43:25.695 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "", 2024-09-13T23:43:25.695 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "" 2024-09-13T23:43:25.695 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:25.695 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_pools": [ 2024-09-13T23:43:25.695 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:25.695 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "default-placement", 2024-09-13T23:43:25.695 INFO:teuthology.orchestra.run.smithi167.stdout: "val": { 2024-09-13T23:43:25.695 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": "default.rgw.buckets.index", 2024-09-13T23:43:25.695 INFO:teuthology.orchestra.run.smithi167.stdout: "storage_classes": { 2024-09-13T23:43:25.695 INFO:teuthology.orchestra.run.smithi167.stdout: "STANDARD": { 2024-09-13T23:43:25.695 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": "default.rgw.buckets.data" 2024-09-13T23:43:25.695 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:25.696 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:25.696 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-09-13T23:43:25.696 INFO:teuthology.orchestra.run.smithi167.stdout: "index_type": 0, 2024-09-13T23:43:25.696 INFO:teuthology.orchestra.run.smithi167.stdout: "inline_data": true 2024-09-13T23:43:25.696 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:25.696 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:25.696 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:25.696 INFO:teuthology.orchestra.run.smithi167.stdout: "realm_id": "", 2024-09-13T23:43:25.696 INFO:teuthology.orchestra.run.smithi167.stdout: "notif_pool": "default.rgw.log:notif" 2024-09-13T23:43:25.696 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:25.697 DEBUG:tasks.util.rgw: json result: {'id': '9144208c-3c85-4fb6-9007-2693d5a930f0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-09-13T23:43:25.699 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-09-13T23:43:25.699 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-09-13T23:43:25.699 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-09-13T23:43:25.738 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:25.738 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:25.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b7736ff40 1 Processor -- start 2024-09-13T23:43:25.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b7736ff40 1 -- start start 2024-09-13T23:43:25.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b7736ff40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55997ebf3000 0x55997eed8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b7736ff40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55997ecb7500 con 0x55997ebf3400 2024-09-13T23:43:25.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b7736ff40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55997ec80a00 con 0x55997ebf3000 2024-09-13T23:43:25.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b75b28700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55997ebf3000 0x55997eed8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b75b28700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55997ebf3000 0x55997eed8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43260/0 (socket says 172.21.15.167:43260) 2024-09-13T23:43:25.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b75327700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55997ebf3400 0x55997ecf2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:40346/0 (socket says 172.21.15.167:40346) 2024-09-13T23:43:25.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b75b28700 1 -- 172.21.15.167:0/3644330606 learned_addr learned my addr 172.21.15.167:0/3644330606 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b75b28700 1 -- 172.21.15.167:0/3644330606 >> v1:172.21.15.12:6789/0 conn(0x55997ebf3400 legacy=0x55997ecf2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b75b28700 1 -- 172.21.15.167:0/3644330606 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55997ec05d40 con 0x55997ebf3000 2024-09-13T23:43:25.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b75b28700 1 --2- 172.21.15.167:0/3644330606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55997eed3a70 tx=0x55997ecf7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c970a4b6dedeae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b74b26700 1 -- 172.21.15.167:0/3644330606 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997ec974a0 con 0x55997ebf3000 2024-09-13T23:43:25.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b74b26700 1 -- 172.21.15.167:0/3644330606 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55997ecb7800 con 0x55997ebf3000 2024-09-13T23:43:25.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b74b26700 1 -- 172.21.15.167:0/3644330606 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997ec97600 con 0x55997ebf3000 2024-09-13T23:43:25.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3644330606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 msgr2=0x55997eed8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3644330606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x55997eed3a70 tx=0x55997ecf7920 comp rx=0 tx=0).stop 2024-09-13T23:43:25.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3644330606 shutdown_connections 2024-09-13T23:43:25.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3644330606 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3644330606 >> 172.21.15.167:0/3644330606 conn(0x55997ebf2c00 msgr2=0x55997ecf2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.731+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3644330606 shutdown_connections 2024-09-13T23:43:25.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3644330606 wait complete. 2024-09-13T23:43:25.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 Processor -- start 2024-09-13T23:43:25.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 -- start start 2024-09-13T23:43:25.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997ec80500 con 0x55997ebf3000 2024-09-13T23:43:25.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b75b28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b75b28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43272/0 (socket says 172.21.15.167:43272) 2024-09-13T23:43:25.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b75b28700 1 -- 172.21.15.167:0/377522203 learned_addr learned my addr 172.21.15.167:0/377522203 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b75b28700 1 -- 172.21.15.167:0/377522203 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55997ec05ba0 con 0x55997ebf3000 2024-09-13T23:43:25.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b75b28700 1 --2- 172.21.15.167:0/377522203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55997eed2c30 tx=0x55997ecf7800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e7a39b13b0837 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b74325700 1 -- 172.21.15.167:0/377522203 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997ec97760 con 0x55997ebf3000 2024-09-13T23:43:25.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b74325700 1 -- 172.21.15.167:0/377522203 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55997ecb7b00 con 0x55997ebf3000 2024-09-13T23:43:25.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b74325700 1 -- 172.21.15.167:0/377522203 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997ec974a0 con 0x55997ebf3000 2024-09-13T23:43:25.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 -- 172.21.15.167:0/377522203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 msgr2=0x55997eed8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/377522203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x55997eed2c30 tx=0x55997ecf7800 comp rx=0 tx=0).stop 2024-09-13T23:43:25.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 -- 172.21.15.167:0/377522203 shutdown_connections 2024-09-13T23:43:25.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/377522203 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 -- 172.21.15.167:0/377522203 >> 172.21.15.167:0/377522203 conn(0x55997ebf2c00 msgr2=0x55997ecf3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 -- 172.21.15.167:0/377522203 shutdown_connections 2024-09-13T23:43:25.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 -- 172.21.15.167:0/377522203 wait complete. 2024-09-13T23:43:25.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 Processor -- start 2024-09-13T23:43:25.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 -- start start 2024-09-13T23:43:25.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.735+0000 7f7b7736ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997ec803c0 con 0x55997ebf3000 2024-09-13T23:43:25.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b75b28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b75b28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43274/0 (socket says 172.21.15.167:43274) 2024-09-13T23:43:25.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b75b28700 1 -- 172.21.15.167:0/23030804 learned_addr learned my addr 172.21.15.167:0/23030804 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b75b28700 1 -- 172.21.15.167:0/23030804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55997ec05a00 con 0x55997ebf3000 2024-09-13T23:43:25.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b75b28700 1 --2- 172.21.15.167:0/23030804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55997ed4ffb0 tx=0x55997ecf7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b72b22700 1 -- 172.21.15.167:0/23030804 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997ec97600 con 0x55997ebf3000 2024-09-13T23:43:25.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b72b22700 1 -- 172.21.15.167:0/23030804 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55997ecb7c80 con 0x55997ebf3000 2024-09-13T23:43:25.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b7736ff40 1 -- 172.21.15.167:0/23030804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55997ec05860 con 0x55997ebf3000 2024-09-13T23:43:25.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b72b22700 1 -- 172.21.15.167:0/23030804 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997ec978c0 con 0x55997ebf3000 2024-09-13T23:43:25.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b7736ff40 1 -- 172.21.15.167:0/23030804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55997ec056c0 con 0x55997ebf3000 2024-09-13T23:43:25.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b72b22700 1 -- 172.21.15.167:0/23030804 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55997ececb00 con 0x55997ebf3000 2024-09-13T23:43:25.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b7736ff40 1 Processor -- start 2024-09-13T23:43:25.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b72b22700 1 --2- 172.21.15.167:0/23030804 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997ef8a000 0x55997eed9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b7736ff40 1 -- start start 2024-09-13T23:43:25.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b72b22700 1 -- 172.21.15.167:0/23030804 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55997ec59180 con 0x55997ebf3000 2024-09-13T23:43:25.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b7736ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 0x55997eed8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b7736ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997ec80000 con 0x55997ebf3800 2024-09-13T23:43:25.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b76329700 1 --2- 172.21.15.167:0/23030804 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997ef8a000 0x55997eed9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b75b28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 0x55997eed8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b75b28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 0x55997eed8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43286/0 (socket says 172.21.15.167:43286) 2024-09-13T23:43:25.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b75b28700 1 -- 172.21.15.167:0/1166510392 learned_addr learned my addr 172.21.15.167:0/1166510392 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.739+0000 7f7b75b28700 1 -- 172.21.15.167:0/1166510392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55997ec04000 con 0x55997ebf3800 2024-09-13T23:43:25.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b75b28700 1 --2- 172.21.15.167:0/1166510392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 0x55997eed8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55997ef1af90 tx=0x55997ef54b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a37239f2b512ac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b76329700 1 --2- 172.21.15.167:0/23030804 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997ef8a000 0x55997eed9080 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55997ef8c630 tx=0x55997ef54e40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7031d700 1 -- 172.21.15.167:0/1166510392 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997ec97600 con 0x55997ebf3800 2024-09-13T23:43:25.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7031d700 1 -- 172.21.15.167:0/1166510392 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55997ecb7c80 con 0x55997ebf3800 2024-09-13T23:43:25.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7031d700 1 -- 172.21.15.167:0/1166510392 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997ec978c0 con 0x55997ebf3800 2024-09-13T23:43:25.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 -- 172.21.15.167:0/1166510392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 msgr2=0x55997eed8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/1166510392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 0x55997eed8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55997ef1af90 tx=0x55997ef54b40 comp rx=0 tx=0).stop 2024-09-13T23:43:25.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 -- 172.21.15.167:0/1166510392 shutdown_connections 2024-09-13T23:43:25.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/1166510392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 0x55997eed8b00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 -- 172.21.15.167:0/1166510392 >> 172.21.15.167:0/1166510392 conn(0x55997ebf3400 msgr2=0x55997ecf3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 -- 172.21.15.167:0/1166510392 shutdown_connections 2024-09-13T23:43:25.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 -- 172.21.15.167:0/1166510392 wait complete. 2024-09-13T23:43:25.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 Processor -- start 2024-09-13T23:43:25.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 -- start start 2024-09-13T23:43:25.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 0x55997eed8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997ec18640 con 0x55997ebf3800 2024-09-13T23:43:25.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b75b28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 0x55997eed8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b75b28700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 0x55997eed8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43296/0 (socket says 172.21.15.167:43296) 2024-09-13T23:43:25.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 learned_addr learned my addr 172.21.15.167:0/3845212519 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55997ec04ea0 con 0x55997ebf3800 2024-09-13T23:43:25.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b75b28700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 0x55997eed8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55997ef8d050 tx=0x55997ef54d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b6eb1a700 1 -- 172.21.15.167:0/3845212519 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997ec97760 con 0x55997ebf3800 2024-09-13T23:43:25.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55997ec04820 con 0x55997ebf3800 2024-09-13T23:43:25.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b6eb1a700 1 -- 172.21.15.167:0/3845212519 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55997efb2000 con 0x55997ebf3800 2024-09-13T23:43:25.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b6eb1a700 1 -- 172.21.15.167:0/3845212519 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997ec974a0 con 0x55997ebf3800 2024-09-13T23:43:25.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.743+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55997ec049c0 con 0x55997ebf3800 2024-09-13T23:43:25.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b6eb1a700 1 -- 172.21.15.167:0/3845212519 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55997ececdc0 con 0x55997ebf3800 2024-09-13T23:43:25.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b6eb1a700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997ef8b000 0x55997eed9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b6eb1a700 1 -- 172.21.15.167:0/3845212519 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55997ec59340 con 0x55997ebf3800 2024-09-13T23:43:25.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b75327700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997ef8b000 0x55997eed9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b75327700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997ef8b000 0x55997eed9600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55997eed3f80 tx=0x55997f012000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b7736ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55997ef8b400 0x55997eed9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997ef8b800 con 0x55997ef8b400 2024-09-13T23:43:25.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b76329700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55997ef8b400 0x55997eed9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b76329700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55997ef8b400 0x55997eed9b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b7736ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b7736ff40 20 realm 2024-09-13T23:43:25.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b7736ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997ef8bc00 con 0x55997ef8b400 2024-09-13T23:43:25.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b7736ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b7736ff40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:25.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.747+0000 7f7b7736ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997ef8b800 con 0x55997ef8b400 2024-09-13T23:43:25.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b7736ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b7736ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55997ef8bc00 0x55997eeda100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028000 con 0x55997ef8bc00 2024-09-13T23:43:25.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b75b28700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55997ef8bc00 0x55997eeda100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b75b28700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55997ef8bc00 0x55997eeda100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b7736ff40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:25.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b7736ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8bc00 2024-09-13T23:43:25.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b7736ff40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:25.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b7736ff40 20 searching for the correct realm 2024-09-13T23:43:25.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55997f028800 0x55997eeda680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028c00 con 0x55997f028800 2024-09-13T23:43:25.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.751+0000 7f7b75327700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55997f028800 0x55997eeda680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b75327700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55997f028800 0x55997eeda680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028c00 con 0x55997ef8bc00 2024-09-13T23:43:25.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b75b28700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55997f028000 0x55997eedac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028000 2024-09-13T23:43:25.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b76329700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55997f028000 0x55997eedac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b76329700 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55997f028000 0x55997eedac00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8b400 2024-09-13T23:43:25.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8b400 2024-09-13T23:43:25.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.755+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028800 2024-09-13T23:43:25.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8b400 2024-09-13T23:43:25.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8bc00 2024-09-13T23:43:25.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8bc00 2024-09-13T23:43:25.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8b400 2024-09-13T23:43:25.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028000 2024-09-13T23:43:25.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8b400 2024-09-13T23:43:25.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.759+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028000 2024-09-13T23:43:25.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.763+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.763+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8bc00 2024-09-13T23:43:25.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.763+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.763+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028800 2024-09-13T23:43:25.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.763+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.763+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8b400 2024-09-13T23:43:25.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.763+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.763+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8bc00 2024-09-13T23:43:25.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.763+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.763+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028800 2024-09-13T23:43:25.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.763+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.763+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8b400 2024-09-13T23:43:25.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8b400 2024-09-13T23:43:25.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8b400 2024-09-13T23:43:25.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028800 2024-09-13T23:43:25.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028000 2024-09-13T23:43:25.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8b400 2024-09-13T23:43:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028000 2024-09-13T23:43:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028800 2024-09-13T23:43:25.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.767+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.771+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028000 2024-09-13T23:43:25.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.771+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.771+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8bc00 2024-09-13T23:43:25.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.771+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.771+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8bc00 2024-09-13T23:43:25.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.771+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.771+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028800 2024-09-13T23:43:25.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.771+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.771+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8bc00 2024-09-13T23:43:25.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.771+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.771+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8bc00 2024-09-13T23:43:25.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:25.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:25.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:25.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:25.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029000 con 0x55997ef8b400 2024-09-13T23:43:25.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029400 con 0x55997ef8bc00 2024-09-13T23:43:25.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:25.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029800 con 0x55997ef8bc00 2024-09-13T23:43:25.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:25.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 zone default found 2024-09-13T23:43:25.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 4 Realm: () 2024-09-13T23:43:25.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:25.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:25.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:25.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 zonegroup default 2024-09-13T23:43:25.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029c00 con 0x55997f028000 2024-09-13T23:43:25.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.775+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029000 con 0x55997f028800 2024-09-13T23:43:25.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.779+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.779+0000 7f7b7736ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.779+0000 7f7b7736ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.779+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029c00 con 0x55997f028800 2024-09-13T23:43:25.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.779+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.779+0000 7f7b7736ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.779+0000 7f7b7736ff40 20 started sync module instance, tier type = 2024-09-13T23:43:25.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.779+0000 7f7b7736ff40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:25.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.779+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997ef8b800 con 0x55997ef8bc00 2024-09-13T23:43:25.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.779+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.779+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94117747817216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028c00 con 0x55997ef8bc00 2024-09-13T23:43:25.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.783+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8b400 2024-09-13T23:43:25.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.783+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94117747817216] v33'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.783+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.783+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94117747818368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029400 con 0x55997ef8b400 2024-09-13T23:43:25.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.783+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029800 con 0x55997ef8bc00 2024-09-13T23:43:25.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.787+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94117747818368] v33'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.787+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.787+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94117747819520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029000 con 0x55997ef8bc00 2024-09-13T23:43:25.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.787+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029c00 con 0x55997f028000 2024-09-13T23:43:25.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.791+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.791+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94117747820672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997ef8a800 con 0x55997f028000 2024-09-13T23:43:25.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.791+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997ef8b800 con 0x55997ef8bc00 2024-09-13T23:43:25.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.791+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94117747819520] v33'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.791+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94117747820672] v33'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.791+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.791+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94117747821824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997ef8a800 con 0x55997ef8bc00 2024-09-13T23:43:25.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.791+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028800 2024-09-13T23:43:25.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.795+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.795+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94117747822976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029400 con 0x55997f028800 2024-09-13T23:43:25.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.795+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029c00 con 0x55997f028800 2024-09-13T23:43:25.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.795+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94117747821824] v33'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.795+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.795+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94117747824128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028800 2024-09-13T23:43:25.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.795+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029400 con 0x55997f028000 2024-09-13T23:43:25.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.795+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94117747822976] v33'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94117747824128] v33'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'18 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94117747825280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029c00 con 0x55997f028000 2024-09-13T23:43:25.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b7736ff40 20 add_watcher() i=0 2024-09-13T23:43:25.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b7736ff40 20 add_watcher() i=1 2024-09-13T23:43:25.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b7736ff40 20 add_watcher() i=2 2024-09-13T23:43:25.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b7736ff40 20 add_watcher() i=3 2024-09-13T23:43:25.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b7736ff40 20 add_watcher() i=4 2024-09-13T23:43:25.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b7736ff40 20 add_watcher() i=5 2024-09-13T23:43:25.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b7736ff40 20 add_watcher() i=6 2024-09-13T23:43:25.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94117747825280] v33'19 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b7736ff40 20 add_watcher() i=7 2024-09-13T23:43:25.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b7736ff40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:25.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f036000 con 0x55997ef8b400 2024-09-13T23:43:25.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.799+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94117749243904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f036400 con 0x55997ef8b400 2024-09-13T23:43:25.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.803+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94117749243904] v33'46 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.803+0000 7f7b5caf6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:25.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.803+0000 7f7b7736ff40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:25.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.803+0000 7f7b7736ff40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:25.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.803+0000 7f7b7736ff40 1 Processor -- start 2024-09-13T23:43:25.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.803+0000 7f7b7736ff40 1 -- start start 2024-09-13T23:43:25.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.803+0000 7f7b7736ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f036c00 0x55997eedb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.803+0000 7f7b7736ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997ec18780 con 0x55997f036c00 2024-09-13T23:43:25.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.803+0000 7f7b75327700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f036c00 0x55997eedb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.803+0000 7f7b75327700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f036c00 0x55997eedb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43308/0 (socket says 172.21.15.167:43308) 2024-09-13T23:43:25.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.803+0000 7f7b75327700 1 -- 172.21.15.167:0/2802356723 learned_addr learned my addr 172.21.15.167:0/2802356723 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.803+0000 7f7b75327700 1 -- 172.21.15.167:0/2802356723 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55997ec04b60 con 0x55997f036c00 2024-09-13T23:43:25.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b75327700 1 --2- 172.21.15.167:0/2802356723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f036c00 0x55997eedb180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55997f042060 tx=0x55997ef55f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be6f2511ec1cc46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b5c2f5700 1 -- 172.21.15.167:0/2802356723 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997f038160 con 0x55997f036c00 2024-09-13T23:43:25.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b5c2f5700 1 -- 172.21.15.167:0/2802356723 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55997efb2a80 con 0x55997f036c00 2024-09-13T23:43:25.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b5c2f5700 1 -- 172.21.15.167:0/2802356723 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997f0382c0 con 0x55997f036c00 2024-09-13T23:43:25.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b7736ff40 1 -- 172.21.15.167:0/2802356723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f036c00 msgr2=0x55997eedb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/2802356723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f036c00 0x55997eedb180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55997f042060 tx=0x55997ef55f20 comp rx=0 tx=0).stop 2024-09-13T23:43:25.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b7736ff40 1 -- 172.21.15.167:0/2802356723 shutdown_connections 2024-09-13T23:43:25.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/2802356723 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f036c00 0x55997eedb180 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b7736ff40 1 -- 172.21.15.167:0/2802356723 >> 172.21.15.167:0/2802356723 conn(0x55997f036800 msgr2=0x55997ecf4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b7736ff40 1 -- 172.21.15.167:0/2802356723 shutdown_connections 2024-09-13T23:43:25.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b7736ff40 1 -- 172.21.15.167:0/2802356723 wait complete. 2024-09-13T23:43:25.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b7736ff40 1 Processor -- start 2024-09-13T23:43:25.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b7736ff40 1 -- start start 2024-09-13T23:43:25.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b7736ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f037000 0x55997eedb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b7736ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55997ec80000 con 0x55997f037000 2024-09-13T23:43:25.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b75327700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f037000 0x55997eedb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b75327700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f037000 0x55997eedb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43322/0 (socket says 172.21.15.167:43322) 2024-09-13T23:43:25.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b75327700 1 -- 172.21.15.167:0/3313545600 learned_addr learned my addr 172.21.15.167:0/3313545600 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b75327700 1 -- 172.21.15.167:0/3313545600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55997ec04d00 con 0x55997f037000 2024-09-13T23:43:25.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.807+0000 7f7b75327700 1 --2- 172.21.15.167:0/3313545600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f037000 0x55997eedb700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55997f042270 tx=0x55997f0120c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b5aaf2700 1 -- 172.21.15.167:0/3313545600 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997f038420 con 0x55997f037000 2024-09-13T23:43:25.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b5aaf2700 1 -- 172.21.15.167:0/3313545600 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55997efb2780 con 0x55997f037000 2024-09-13T23:43:25.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3313545600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55997ec04340 con 0x55997f037000 2024-09-13T23:43:25.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b5aaf2700 1 -- 172.21.15.167:0/3313545600 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55997f038160 con 0x55997f037000 2024-09-13T23:43:25.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3313545600 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55997ec04b60 con 0x55997f037000 2024-09-13T23:43:25.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b5aaf2700 1 -- 172.21.15.167:0/3313545600 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55997eced8c0 con 0x55997f037000 2024-09-13T23:43:25.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b5aaf2700 1 --2- 172.21.15.167:0/3313545600 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997f0ca000 0x55997f0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b5aaf2700 1 -- 172.21.15.167:0/3313545600 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55997ec596c0 con 0x55997f037000 2024-09-13T23:43:25.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b7736ff40 5 note: GC not initialized 2024-09-13T23:43:25.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b76329700 1 --2- 172.21.15.167:0/3313545600 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997f0ca000 0x55997f0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b76329700 1 --2- 172.21.15.167:0/3313545600 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997f0ca000 0x55997f0d0000 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55997efc99b0 tx=0x55997ef54a80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b582ed700 20 reqs_thread_entry: start 2024-09-13T23:43:25.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.811+0000 7f7b7736ff40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:25.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.815+0000 7f7b7736ff40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:25.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.815+0000 7f7b562e9700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:56 2024 2024-09-13T23:43:25.823 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:25.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.815+0000 7f7b55ac7700 20 reqs_thread_entry: start 2024-09-13T23:43:25.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.815+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f037c00 con 0x55997ef8bc00 2024-09-13T23:43:25.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.815+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.815+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0ca400 con 0x55997ef8bc00 2024-09-13T23:43:25.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.815+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.815+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0ca800 con 0x55997ef8bc00 2024-09-13T23:43:25.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.815+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.815+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cac00 con 0x55997f028800 2024-09-13T23:43:25.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.815+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.815+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cb000 con 0x55997ef8bc00 2024-09-13T23:43:25.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cb400 con 0x55997f028800 2024-09-13T23:43:25.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cb800 con 0x55997ef8b400 2024-09-13T23:43:25.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cbc00 con 0x55997ef8b400 2024-09-13T23:43:25.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cb800 con 0x55997f028800 2024-09-13T23:43:25.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0ca400 con 0x55997f028800 2024-09-13T23:43:25.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f036000 con 0x55997f028800 2024-09-13T23:43:25.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(72 usage.10 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.819+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f036400 con 0x55997ef8b400 2024-09-13T23:43:25.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f037c00 con 0x55997f028000 2024-09-13T23:43:25.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997ef8b800 con 0x55997f028800 2024-09-13T23:43:25.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997ef8a800 con 0x55997f028000 2024-09-13T23:43:25.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029400 con 0x55997ef8b400 2024-09-13T23:43:25.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029c00 con 0x55997f028800 2024-09-13T23:43:25.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028c00 con 0x55997ef8b400 2024-09-13T23:43:25.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(79 usage.17 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.823+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029800 con 0x55997ef8bc00 2024-09-13T23:43:25.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.827+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.827+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029000 con 0x55997ef8bc00 2024-09-13T23:43:25.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.827+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.827+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cbc00 con 0x55997f028800 2024-09-13T23:43:25.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.827+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.827+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cac00 con 0x55997ef8bc00 2024-09-13T23:43:25.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.827+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.827+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cb400 con 0x55997f028800 2024-09-13T23:43:25.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.827+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.827+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cb800 con 0x55997ef8b400 2024-09-13T23:43:25.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.827+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.827+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029400 con 0x55997ef8bc00 2024-09-13T23:43:25.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.831+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.831+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028c00 con 0x55997ef8bc00 2024-09-13T23:43:25.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.831+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.831+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997ef8bc00 2024-09-13T23:43:25.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.831+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.831+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997ef8a800 con 0x55997ef8b400 2024-09-13T23:43:25.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.831+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.831+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f036400 con 0x55997f028800 2024-09-13T23:43:25.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.831+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.831+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f037c00 con 0x55997f028800 2024-09-13T23:43:25.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.831+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.831+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0ca800 con 0x55997ef8bc00 2024-09-13T23:43:25.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.835+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.835+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cb000 con 0x55997ef8b400 2024-09-13T23:43:25.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.835+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.835+0000 7f7b5caf6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:25.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.835+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94117749243904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cac00 con 0x55997ef8b400 2024-09-13T23:43:25.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.839+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94117749243904] v33'47 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.839+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94117747817216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0ca400 con 0x55997ef8bc00 2024-09-13T23:43:25.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.843+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 31 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94117747817216] v33'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.843+0000 7f7b7736ff40 20 remove_watcher() i=0 2024-09-13T23:43:25.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.843+0000 7f7b7736ff40 2 removed watcher, disabling cache 2024-09-13T23:43:25.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.843+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94117747818368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f0cbc00 con 0x55997ef8b400 2024-09-13T23:43:25.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.843+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.1 v2:172.21.15.167:6800/3875661733 28 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94117747818368] v33'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997ef8b400 2024-09-13T23:43:25.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.843+0000 7f7b7736ff40 20 remove_watcher() i=1 2024-09-13T23:43:25.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.843+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94117747819520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029800 con 0x55997ef8bc00 2024-09-13T23:43:25.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.847+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 32 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94117747819520] v33'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.847+0000 7f7b7736ff40 20 remove_watcher() i=2 2024-09-13T23:43:25.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.847+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94117747820672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029000 con 0x55997f028000 2024-09-13T23:43:25.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.851+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94117747820672] v33'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.851+0000 7f7b7736ff40 20 remove_watcher() i=3 2024-09-13T23:43:25.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.851+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94117747821824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029400 con 0x55997ef8bc00 2024-09-13T23:43:25.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.851+0000 7f7b75b28700 1 -- 172.21.15.167:0/3845212519 <== osd.0 v2:172.21.15.12:6800/2649399468 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94117747821824] v33'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023200 con 0x55997ef8bc00 2024-09-13T23:43:25.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.851+0000 7f7b7736ff40 20 remove_watcher() i=4 2024-09-13T23:43:25.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.851+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94117747822976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028c00 con 0x55997f028800 2024-09-13T23:43:25.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.855+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94117747822976] v33'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.855+0000 7f7b7736ff40 20 remove_watcher() i=5 2024-09-13T23:43:25.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.855+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94117747824128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f028400 con 0x55997f028800 2024-09-13T23:43:25.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.855+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 <== osd.3 v2:172.21.15.167:6816/3688363524 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94117747824128] v33'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f023440 con 0x55997f028800 2024-09-13T23:43:25.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.855+0000 7f7b7736ff40 20 remove_watcher() i=6 2024-09-13T23:43:25.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.855+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94117747825280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55997f029c00 con 0x55997f028000 2024-09-13T23:43:25.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b76329700 1 -- 172.21.15.167:0/3845212519 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94117747825280] v33'20 uv18 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55997f022b40 con 0x55997f028000 2024-09-13T23:43:25.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 20 remove_watcher() i=7 2024-09-13T23:43:25.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55997ef8bc00 msgr2=0x55997eeda100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55997ef8bc00 0x55997eeda100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55997ef8b400 msgr2=0x55997eed9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55997ef8b400 0x55997eed9b80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55997f028000 msgr2=0x55997eedac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55997f028000 0x55997eedac00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55997f028800 msgr2=0x55997eeda680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55997f028800 0x55997eeda680 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997ef8b000 msgr2=0x55997eed9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997ef8b000 0x55997eed9600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x55997eed3f80 tx=0x55997f012000 comp rx=0 tx=0).stop 2024-09-13T23:43:25.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 msgr2=0x55997eed8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3845212519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3800 0x55997eed8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55997ef8d050 tx=0x55997ef54d20 comp rx=0 tx=0).stop 2024-09-13T23:43:25.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 reap_dead start 2024-09-13T23:43:25.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b75327700 1 -- 172.21.15.167:0/3845212519 reap_dead start 2024-09-13T23:43:25.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 shutdown_connections 2024-09-13T23:43:25.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 >> 172.21.15.167:0/3845212519 conn(0x55997ebf3400 msgr2=0x55997ecf3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 shutdown_connections 2024-09-13T23:43:25.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3845212519 wait complete. 2024-09-13T23:43:25.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3313545600 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997f0ca000 msgr2=0x55997f0d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.859+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3313545600 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997f0ca000 0x55997f0d0000 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55997efc99b0 tx=0x55997ef54a80 comp rx=0 tx=0).stop 2024-09-13T23:43:25.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3313545600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f037000 msgr2=0x55997eedb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3313545600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f037000 0x55997eedb700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55997f042270 tx=0x55997f0120c0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3313545600 shutdown_connections 2024-09-13T23:43:25.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3313545600 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997f0ca000 0x55997f0d0000 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/3313545600 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997f037000 0x55997eedb700 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3313545600 >> 172.21.15.167:0/3313545600 conn(0x55997f036800 msgr2=0x55997ecf4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3313545600 shutdown_connections 2024-09-13T23:43:25.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 -- 172.21.15.167:0/3313545600 wait complete. 2024-09-13T23:43:25.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 -- 172.21.15.167:0/23030804 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997ef8a000 msgr2=0x55997eed9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/23030804 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997ef8a000 0x55997eed9080 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55997ef8c630 tx=0x55997ef54e40 comp rx=0 tx=0).stop 2024-09-13T23:43:25.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 -- 172.21.15.167:0/23030804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 msgr2=0x55997eed8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/23030804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55997ed4ffb0 tx=0x55997ecf7920 comp rx=0 tx=0).stop 2024-09-13T23:43:25.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 -- 172.21.15.167:0/23030804 shutdown_connections 2024-09-13T23:43:25.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/23030804 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55997ef8a000 0x55997eed9080 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 --2- 172.21.15.167:0/23030804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55997ebf3000 0x55997eed8580 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 -- 172.21.15.167:0/23030804 >> 172.21.15.167:0/23030804 conn(0x55997ebf2c00 msgr2=0x55997ecf3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 -- 172.21.15.167:0/23030804 shutdown_connections 2024-09-13T23:43:25.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.863+0000 7f7b7736ff40 1 -- 172.21.15.167:0/23030804 wait complete. 2024-09-13T23:43:25.875 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:25.875 INFO:teuthology.orchestra.run.smithi167.stdout: "entries": [], 2024-09-13T23:43:25.875 INFO:teuthology.orchestra.run.smithi167.stdout: "summary": [] 2024-09-13T23:43:25.875 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:25.877 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-09-13T23:43:25.878 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-13T23:43:25.878 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-13T23:43:25.878 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-13T23:43:25.961 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:25.962 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:25.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.951+0000 7f26109d2f40 1 Processor -- start 2024-09-13T23:43:25.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 -- start start 2024-09-13T23:43:25.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55653c7d1400 0x55653cab6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55653c895500 con 0x55653c7d1000 2024-09-13T23:43:25.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55653c85ea00 con 0x55653c7d1400 2024-09-13T23:43:25.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f260e98a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55653c7d1400 0x55653cab6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f260e98a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55653c7d1400 0x55653cab6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43326/0 (socket says 172.21.15.167:43326) 2024-09-13T23:43:25.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f260f18b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55653c7d1000 0x55653c8d0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:40362/0 (socket says 172.21.15.167:40362) 2024-09-13T23:43:25.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f260e98a700 1 -- 172.21.15.167:0/2072039432 learned_addr learned my addr 172.21.15.167:0/2072039432 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f260e98a700 1 -- 172.21.15.167:0/2072039432 >> v1:172.21.15.12:6789/0 conn(0x55653c7d1000 legacy=0x55653c8d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f260e98a700 1 -- 172.21.15.167:0/2072039432 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653c7e3d40 con 0x55653c7d1400 2024-09-13T23:43:25.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f260e98a700 1 --2- 172.21.15.167:0/2072039432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1400 0x55653cab6580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55653cab1920 tx=0x55653c8d5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897c0f44b9aecfde server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f260e189700 1 -- 172.21.15.167:0/2072039432 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653c8754a0 con 0x55653c7d1400 2024-09-13T23:43:25.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f260e189700 1 -- 172.21.15.167:0/2072039432 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55653c895800 con 0x55653c7d1400 2024-09-13T23:43:25.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f260e189700 1 -- 172.21.15.167:0/2072039432 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653c875600 con 0x55653c7d1400 2024-09-13T23:43:25.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 -- 172.21.15.167:0/2072039432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1400 msgr2=0x55653cab6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 --2- 172.21.15.167:0/2072039432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1400 0x55653cab6580 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55653cab1920 tx=0x55653c8d5920 comp rx=0 tx=0).stop 2024-09-13T23:43:25.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 -- 172.21.15.167:0/2072039432 shutdown_connections 2024-09-13T23:43:25.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 --2- 172.21.15.167:0/2072039432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1400 0x55653cab6580 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 -- 172.21.15.167:0/2072039432 >> 172.21.15.167:0/2072039432 conn(0x55653c7d0c00 msgr2=0x55653c8d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 -- 172.21.15.167:0/2072039432 shutdown_connections 2024-09-13T23:43:25.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 -- 172.21.15.167:0/2072039432 wait complete. 2024-09-13T23:43:25.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 Processor -- start 2024-09-13T23:43:25.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.955+0000 7f26109d2f40 1 -- start start 2024-09-13T23:43:25.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 0x55653cab6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653c85e500 con 0x55653c7d1000 2024-09-13T23:43:25.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f260f18b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 0x55653cab6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f260f18b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 0x55653cab6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43342/0 (socket says 172.21.15.167:43342) 2024-09-13T23:43:25.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f260f18b700 1 -- 172.21.15.167:0/2899871039 learned_addr learned my addr 172.21.15.167:0/2899871039 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f260f18b700 1 -- 172.21.15.167:0/2899871039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653c7e3ba0 con 0x55653c7d1000 2024-09-13T23:43:25.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f260f18b700 1 --2- 172.21.15.167:0/2899871039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 0x55653cab6580 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55653c92de90 tx=0x55653c8d5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480ea383d1e8df05 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f260d988700 1 -- 172.21.15.167:0/2899871039 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653c8754a0 con 0x55653c7d1000 2024-09-13T23:43:25.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f260d988700 1 -- 172.21.15.167:0/2899871039 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55653c895b00 con 0x55653c7d1000 2024-09-13T23:43:25.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f260d988700 1 -- 172.21.15.167:0/2899871039 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653c875600 con 0x55653c7d1000 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 -- 172.21.15.167:0/2899871039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 msgr2=0x55653cab6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 --2- 172.21.15.167:0/2899871039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 0x55653cab6580 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55653c92de90 tx=0x55653c8d5b60 comp rx=0 tx=0).stop 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 -- 172.21.15.167:0/2899871039 shutdown_connections 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 --2- 172.21.15.167:0/2899871039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 0x55653cab6580 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 -- 172.21.15.167:0/2899871039 >> 172.21.15.167:0/2899871039 conn(0x55653c7d0c00 msgr2=0x55653c8d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 -- 172.21.15.167:0/2899871039 shutdown_connections 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 -- 172.21.15.167:0/2899871039 wait complete. 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 Processor -- start 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 -- start start 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 0x55653cab6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f26109d2f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653c85e3c0 con 0x55653c7d1000 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f260f18b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 0x55653cab6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f260f18b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 0x55653cab6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43352/0 (socket says 172.21.15.167:43352) 2024-09-13T23:43:25.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.959+0000 7f260f18b700 1 -- 172.21.15.167:0/888115354 learned_addr learned my addr 172.21.15.167:0/888115354 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260f18b700 1 -- 172.21.15.167:0/888115354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653c7e3a00 con 0x55653c7d1000 2024-09-13T23:43:25.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260f18b700 1 --2- 172.21.15.167:0/888115354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 0x55653cab6580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55653d6420f0 tx=0x55653c8d5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260c185700 1 -- 172.21.15.167:0/888115354 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653c8758c0 con 0x55653c7d1000 2024-09-13T23:43:25.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260c185700 1 -- 172.21.15.167:0/888115354 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55653c895c80 con 0x55653c7d1000 2024-09-13T23:43:25.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260c185700 1 -- 172.21.15.167:0/888115354 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653c8754a0 con 0x55653c7d1000 2024-09-13T23:43:25.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f26109d2f40 1 -- 172.21.15.167:0/888115354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55653c7e3860 con 0x55653c7d1000 2024-09-13T23:43:25.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f26109d2f40 1 -- 172.21.15.167:0/888115354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55653c7e36c0 con 0x55653c7d1000 2024-09-13T23:43:25.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f26109d2f40 1 Processor -- start 2024-09-13T23:43:25.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f26109d2f40 1 -- start start 2024-09-13T23:43:25.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f26109d2f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 0x55653cab6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f26109d2f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653c85e000 con 0x55653c7d1800 2024-09-13T23:43:25.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260f98c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 0x55653cab6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260f98c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 0x55653cab6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43360/0 (socket says 172.21.15.167:43360) 2024-09-13T23:43:25.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260f98c700 1 -- 172.21.15.167:0/4018900033 learned_addr learned my addr 172.21.15.167:0/4018900033 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260c185700 1 -- 172.21.15.167:0/888115354 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55653c8cab00 con 0x55653c7d1000 2024-09-13T23:43:25.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260f98c700 1 -- 172.21.15.167:0/4018900033 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653c7e2000 con 0x55653c7d1800 2024-09-13T23:43:25.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260c185700 1 --2- 172.21.15.167:0/888115354 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653cb8c000 0x55653cab7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260c185700 1 -- 172.21.15.167:0/888115354 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55653c837180 con 0x55653c7d1000 2024-09-13T23:43:25.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260f98c700 1 --2- 172.21.15.167:0/4018900033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 0x55653cab6b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55653d6433e0 tx=0x55653cb2c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a841637218cccebb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f260f18b700 1 --2- 172.21.15.167:0/888115354 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653cb8c000 0x55653cab7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f2609980700 1 -- 172.21.15.167:0/4018900033 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653c8758c0 con 0x55653c7d1800 2024-09-13T23:43:25.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f2609980700 1 -- 172.21.15.167:0/4018900033 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55653cb40000 con 0x55653c7d1800 2024-09-13T23:43:25.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.963+0000 7f2609980700 1 -- 172.21.15.167:0/4018900033 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653c8754a0 con 0x55653c7d1800 2024-09-13T23:43:25.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 -- 172.21.15.167:0/4018900033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 msgr2=0x55653cab6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:25.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f260f18b700 1 --2- 172.21.15.167:0/888115354 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653cb8c000 0x55653cab7080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55653cb667b0 tx=0x55653cb2c360 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 --2- 172.21.15.167:0/4018900033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 0x55653cab6b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x55653d6433e0 tx=0x55653cb2c960 comp rx=0 tx=0).stop 2024-09-13T23:43:25.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 -- 172.21.15.167:0/4018900033 shutdown_connections 2024-09-13T23:43:25.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 --2- 172.21.15.167:0/4018900033 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 0x55653cab6b00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:25.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 -- 172.21.15.167:0/4018900033 >> 172.21.15.167:0/4018900033 conn(0x55653c7d1400 msgr2=0x55653c8d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:25.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 -- 172.21.15.167:0/4018900033 shutdown_connections 2024-09-13T23:43:25.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 -- 172.21.15.167:0/4018900033 wait complete. 2024-09-13T23:43:25.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 Processor -- start 2024-09-13T23:43:25.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 -- start start 2024-09-13T23:43:25.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 0x55653cab6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653c7f6640 con 0x55653c7d1800 2024-09-13T23:43:25.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f260f98c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 0x55653cab6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f260f98c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 0x55653cab6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43364/0 (socket says 172.21.15.167:43364) 2024-09-13T23:43:25.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 learned_addr learned my addr 172.21.15.167:0/4279797753 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:25.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653c7e2ea0 con 0x55653c7d1800 2024-09-13T23:43:25.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f260f98c700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 0x55653cab6b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55653d642930 tx=0x55653cb2c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f260817d700 1 -- 172.21.15.167:0/4279797753 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653c875b80 con 0x55653c7d1800 2024-09-13T23:43:25.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55653c7e2820 con 0x55653c7d1800 2024-09-13T23:43:25.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f260817d700 1 -- 172.21.15.167:0/4279797753 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55653cb40180 con 0x55653c7d1800 2024-09-13T23:43:25.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f260817d700 1 -- 172.21.15.167:0/4279797753 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653c8758c0 con 0x55653c7d1800 2024-09-13T23:43:25.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.967+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55653c7e29c0 con 0x55653c7d1800 2024-09-13T23:43:25.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f260817d700 1 -- 172.21.15.167:0/4279797753 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55653c8cadc0 con 0x55653c7d1800 2024-09-13T23:43:25.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f260817d700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653cb8cc00 0x55653cab7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f260817d700 1 -- 172.21.15.167:0/4279797753 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55653c837340 con 0x55653c7d1800 2024-09-13T23:43:25.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55653cb8d000 0x55653cab7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d400 con 0x55653cb8d000 2024-09-13T23:43:25.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f260e98a700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653cb8cc00 0x55653cab7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f260f98c700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55653cb8d000 0x55653cab7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f260e98a700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653cb8cc00 0x55653cab7600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55653cab1920 tx=0x55653c8d5800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f260f98c700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55653cb8d000 0x55653cab7b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 20 realm 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d000 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8dc00 con 0x55653cb8d000 2024-09-13T23:43:25.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:25.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:25.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55653cb8d400 0x55653cab8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.971+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d400 2024-09-13T23:43:25.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f260f18b700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55653cb8d400 0x55653cab8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f260f18b700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55653cb8d400 0x55653cab8100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f26109d2f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f26109d2f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8dc00 con 0x55653cb8d400 2024-09-13T23:43:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f26109d2f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f26109d2f40 20 searching for the correct realm 2024-09-13T23:43:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f26109d2f40 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55653caf4000 0x55653cab8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf4400 con 0x55653caf4000 2024-09-13T23:43:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f260e98a700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55653caf4000 0x55653cab8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f260e98a700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55653caf4000 0x55653cab8680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:25.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.975+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf4400 con 0x55653cb8d400 2024-09-13T23:43:25.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:25.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260f18b700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55653caf4400 0x55653cab8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:25.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4400 2024-09-13T23:43:25.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260f98c700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55653caf4400 0x55653cab8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:25.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260f98c700 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55653caf4400 0x55653cab8c00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d000 2024-09-13T23:43:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d000 2024-09-13T23:43:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:25.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4000 2024-09-13T23:43:25.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:25.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.979+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d000 2024-09-13T23:43:25.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:25.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d400 2024-09-13T23:43:25.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:25.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d400 2024-09-13T23:43:25.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:25.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d000 2024-09-13T23:43:25.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:25.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4400 2024-09-13T23:43:25.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:25.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d000 2024-09-13T23:43:25.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:25.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4400 2024-09-13T23:43:25.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:25.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.983+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d400 2024-09-13T23:43:25.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:25.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4000 2024-09-13T23:43:25.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:25.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d000 2024-09-13T23:43:25.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:25.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d400 2024-09-13T23:43:25.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:25.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4000 2024-09-13T23:43:25.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:25.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d000 2024-09-13T23:43:25.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:25.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d000 2024-09-13T23:43:25.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:25.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d000 2024-09-13T23:43:25.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.987+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:25.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4000 2024-09-13T23:43:25.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:25.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4400 2024-09-13T23:43:25.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:25.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d000 2024-09-13T23:43:26.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:26.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4400 2024-09-13T23:43:26.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:26.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4000 2024-09-13T23:43:26.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:26.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4400 2024-09-13T23:43:26.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:26.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.991+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d400 2024-09-13T23:43:26.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d400 2024-09-13T23:43:26.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4000 2024-09-13T23:43:26.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:26.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d400 2024-09-13T23:43:26.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d400 2024-09-13T23:43:26.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f26109d2f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:26.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f26109d2f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:26.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f26109d2f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:26.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f26109d2f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:26.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f26109d2f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf4800 con 0x55653cb8d000 2024-09-13T23:43:26.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:26.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f26109d2f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f26109d2f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.995+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf4c00 con 0x55653cb8d400 2024-09-13T23:43:26.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:26.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf5000 con 0x55653cb8d400 2024-09-13T23:43:26.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:26.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 zone default found 2024-09-13T23:43:26.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 4 Realm: () 2024-09-13T23:43:26.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:26.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:26.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:26.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 zonegroup default 2024-09-13T23:43:26.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf5400 con 0x55653caf4400 2024-09-13T23:43:26.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:26.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf5800 con 0x55653caf4000 2024-09-13T23:43:26.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:26.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf5c00 con 0x55653caf4000 2024-09-13T23:43:26.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:26.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 started sync module instance, tier type = 2024-09-13T23:43:26.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:26.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:25.999+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf5800 con 0x55653cb8d400 2024-09-13T23:43:26.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.003+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.003+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93893297888000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf5c00 con 0x55653cb8d400 2024-09-13T23:43:26.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.003+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf4c00 con 0x55653cb8d000 2024-09-13T23:43:26.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.007+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 93893297888000] v33'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.007+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:26.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.007+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93893297889152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf5000 con 0x55653cb8d000 2024-09-13T23:43:26.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.007+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf5800 con 0x55653cb8d400 2024-09-13T23:43:26.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.011+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93893297889152] v33'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:26.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.011+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.011+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93893297890304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8dc00 con 0x55653cb8d400 2024-09-13T23:43:26.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.011+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653caf4400 2024-09-13T23:43:26.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.011+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:26.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.011+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93893297891456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf4800 con 0x55653caf4400 2024-09-13T23:43:26.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.011+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf5400 con 0x55653cb8d400 2024-09-13T23:43:26.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.011+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 93893297890304] v33'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.015+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 93893297891456] v33'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:26.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.015+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.015+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93893297892608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf4c00 con 0x55653cb8d400 2024-09-13T23:43:26.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.015+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf5000 con 0x55653caf4000 2024-09-13T23:43:26.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.015+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:26.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.015+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93893297893760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cc42000 con 0x55653caf4000 2024-09-13T23:43:26.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.015+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cc42400 con 0x55653caf4000 2024-09-13T23:43:26.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.019+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93893297892608] v33'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.019+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93893297893760] v33'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:26.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.019+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:26.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.019+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93893297894912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cc42800 con 0x55653caf4000 2024-09-13T23:43:26.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.019+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cc42c00 con 0x55653caf4400 2024-09-13T23:43:26.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.019+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93893297894912] v33'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:26.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'21 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:26.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93893297896064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cc43000 con 0x55653caf4400 2024-09-13T23:43:26.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f26109d2f40 20 add_watcher() i=0 2024-09-13T23:43:26.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f26109d2f40 20 add_watcher() i=1 2024-09-13T23:43:26.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f26109d2f40 20 add_watcher() i=2 2024-09-13T23:43:26.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f26109d2f40 20 add_watcher() i=3 2024-09-13T23:43:26.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f26109d2f40 20 add_watcher() i=4 2024-09-13T23:43:26.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f26109d2f40 20 add_watcher() i=5 2024-09-13T23:43:26.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f26109d2f40 20 add_watcher() i=6 2024-09-13T23:43:26.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 93893297896064] v33'22 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:26.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f26109d2f40 20 add_watcher() i=7 2024-09-13T23:43:26.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f26109d2f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:26.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cc43400 con 0x55653cb8d000 2024-09-13T23:43:26.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:26.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.023+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93893299568640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cc43800 con 0x55653cb8d000 2024-09-13T23:43:26.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93893299568640] v33'48 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:26.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f25f6159700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:26.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f26109d2f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:26.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f26109d2f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:26.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f26109d2f40 1 Processor -- start 2024-09-13T23:43:26.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f26109d2f40 1 -- start start 2024-09-13T23:43:26.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f26109d2f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 0x55653cab9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f26109d2f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653c7f6780 con 0x55653cc42c00 2024-09-13T23:43:26.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f260e98a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 0x55653cab9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f260e98a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 0x55653cab9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43376/0 (socket says 172.21.15.167:43376) 2024-09-13T23:43:26.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f260e98a700 1 -- 172.21.15.167:0/1612898520 learned_addr learned my addr 172.21.15.167:0/1612898520 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f260e98a700 1 -- 172.21.15.167:0/1612898520 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653c7e2b60 con 0x55653cc42c00 2024-09-13T23:43:26.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f260e98a700 1 --2- 172.21.15.167:0/1612898520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 0x55653cab9180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55653cb67650 tx=0x55653c8d5860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b9c063892639e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f25f5958700 1 -- 172.21.15.167:0/1612898520 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653c875760 con 0x55653cc42c00 2024-09-13T23:43:26.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f25f5958700 1 -- 172.21.15.167:0/1612898520 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55653cb40600 con 0x55653cc42c00 2024-09-13T23:43:26.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.027+0000 7f25f5958700 1 -- 172.21.15.167:0/1612898520 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653cc5c000 con 0x55653cc42c00 2024-09-13T23:43:26.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 -- 172.21.15.167:0/1612898520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 msgr2=0x55653cab9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 --2- 172.21.15.167:0/1612898520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 0x55653cab9180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x55653cb67650 tx=0x55653c8d5860 comp rx=0 tx=0).stop 2024-09-13T23:43:26.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 -- 172.21.15.167:0/1612898520 shutdown_connections 2024-09-13T23:43:26.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 --2- 172.21.15.167:0/1612898520 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 0x55653cab9180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 -- 172.21.15.167:0/1612898520 >> 172.21.15.167:0/1612898520 conn(0x55653cc43c00 msgr2=0x55653c8d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 -- 172.21.15.167:0/1612898520 shutdown_connections 2024-09-13T23:43:26.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 -- 172.21.15.167:0/1612898520 wait complete. 2024-09-13T23:43:26.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 Processor -- start 2024-09-13T23:43:26.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 -- start start 2024-09-13T23:43:26.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 0x55653cab9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653c85e000 con 0x55653cc42c00 2024-09-13T23:43:26.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f260e98a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 0x55653cab9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f260e98a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 0x55653cab9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43378/0 (socket says 172.21.15.167:43378) 2024-09-13T23:43:26.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f260e98a700 1 -- 172.21.15.167:0/3567060306 learned_addr learned my addr 172.21.15.167:0/3567060306 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f260e98a700 1 -- 172.21.15.167:0/3567060306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653c7e2d00 con 0x55653cc42c00 2024-09-13T23:43:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f260e98a700 1 --2- 172.21.15.167:0/3567060306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 0x55653cab9180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55653cc501b0 tx=0x55653cc52300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f25f4155700 1 -- 172.21.15.167:0/3567060306 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653cc5c160 con 0x55653cc42c00 2024-09-13T23:43:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f25f4155700 1 -- 172.21.15.167:0/3567060306 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55653cb40300 con 0x55653cc42c00 2024-09-13T23:43:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 -- 172.21.15.167:0/3567060306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55653c7e2340 con 0x55653cc42c00 2024-09-13T23:43:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f25f4155700 1 -- 172.21.15.167:0/3567060306 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55653cc5c2c0 con 0x55653cc42c00 2024-09-13T23:43:26.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.031+0000 7f26109d2f40 1 -- 172.21.15.167:0/3567060306 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55653c7e3d40 con 0x55653cc42c00 2024-09-13T23:43:26.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f25f4155700 1 -- 172.21.15.167:0/3567060306 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55653c8cbb80 con 0x55653cc42c00 2024-09-13T23:43:26.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f25f4155700 1 --2- 172.21.15.167:0/3567060306 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653ccd0000 0x55653ccd6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f25f4155700 1 -- 172.21.15.167:0/3567060306 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55653c837500 con 0x55653cc42c00 2024-09-13T23:43:26.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f26109d2f40 5 note: GC not initialized 2024-09-13T23:43:26.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f260f98c700 1 --2- 172.21.15.167:0/3567060306 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653ccd0000 0x55653ccd6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f260f98c700 1 --2- 172.21.15.167:0/3567060306 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653ccd0000 0x55653ccd6000 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55653cbd1cb0 tx=0x55653cb2d7a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f25f1950700 20 reqs_thread_entry: start 2024-09-13T23:43:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f26109d2f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f26109d2f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f25ef94c700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:56 2024 2024-09-13T23:43:26.046 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f25ef12a700 20 reqs_thread_entry: start 2024-09-13T23:43:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f26109d2f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.035+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cc42000 con 0x55653cb8d000 2024-09-13T23:43:26.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.039+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:26.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.039+0000 7f26109d2f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.047 INFO:teuthology.orchestra.run.smithi167.stderr:could not fetch user info: no user info saved 2024-09-13T23:43:26.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.039+0000 7f25f6159700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:26.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.039+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93893299568640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cc42400 con 0x55653cb8d000 2024-09-13T23:43:26.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.039+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 93893299568640] v33'49 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:26.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.039+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93893297888000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf4800 con 0x55653cb8d400 2024-09-13T23:43:26.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.043+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 93893297888000] v33'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.043+0000 7f26109d2f40 20 remove_watcher() i=0 2024-09-13T23:43:26.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.043+0000 7f26109d2f40 2 removed watcher, disabling cache 2024-09-13T23:43:26.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.043+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93893297889152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653caf5000 con 0x55653cb8d000 2024-09-13T23:43:26.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.047+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 93893297889152] v33'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653cb8d000 2024-09-13T23:43:26.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.047+0000 7f26109d2f40 20 remove_watcher() i=1 2024-09-13T23:43:26.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.047+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93893297890304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653cb8d800 con 0x55653cb8d400 2024-09-13T23:43:26.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.047+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 93893297890304] v33'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.047+0000 7f26109d2f40 20 remove_watcher() i=2 2024-09-13T23:43:26.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.047+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93893297891456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653ccd0400 con 0x55653caf4400 2024-09-13T23:43:26.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.051+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 93893297891456] v33'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:26.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.051+0000 7f26109d2f40 20 remove_watcher() i=3 2024-09-13T23:43:26.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.051+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93893297892608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653ccd0800 con 0x55653cb8d400 2024-09-13T23:43:26.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.051+0000 7f260f18b700 1 -- 172.21.15.167:0/4279797753 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 93893297892608] v33'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27200 con 0x55653cb8d400 2024-09-13T23:43:26.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.051+0000 7f26109d2f40 20 remove_watcher() i=4 2024-09-13T23:43:26.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.051+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93893297893760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653ccd0c00 con 0x55653caf4000 2024-09-13T23:43:26.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.055+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 93893297893760] v33'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:26.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.055+0000 7f26109d2f40 20 remove_watcher() i=5 2024-09-13T23:43:26.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.055+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93893297894912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653ccd1000 con 0x55653caf4000 2024-09-13T23:43:26.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 93893297894912] v33'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc27440 con 0x55653caf4000 2024-09-13T23:43:26.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 20 remove_watcher() i=6 2024-09-13T23:43:26.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93893297896064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55653ccd1400 con 0x55653caf4400 2024-09-13T23:43:26.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f260f98c700 1 -- 172.21.15.167:0/4279797753 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 93893297896064] v33'23 uv21 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55653cc26b40 con 0x55653caf4400 2024-09-13T23:43:26.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 20 remove_watcher() i=7 2024-09-13T23:43:26.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55653cb8d400 msgr2=0x55653cab8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55653cb8d400 0x55653cab8100 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55653cb8d000 msgr2=0x55653cab7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55653cb8d000 0x55653cab7b80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55653caf4400 msgr2=0x55653cab8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55653caf4400 0x55653cab8c00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55653caf4000 msgr2=0x55653cab8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55653caf4000 0x55653cab8680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653cb8cc00 msgr2=0x55653cab7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653cb8cc00 0x55653cab7600 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55653cab1920 tx=0x55653c8d5800 comp rx=0 tx=0).stop 2024-09-13T23:43:26.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 msgr2=0x55653cab6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f260e98a700 1 -- 172.21.15.167:0/4279797753 reap_dead start 2024-09-13T23:43:26.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 0x55653cab6b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x55653d642930 tx=0x55653cb2c000 comp rx=0 tx=0).stop 2024-09-13T23:43:26.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 shutdown_connections 2024-09-13T23:43:26.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 --2- 172.21.15.167:0/4279797753 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1800 0x55653cab6b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 >> 172.21.15.167:0/4279797753 conn(0x55653c7d1400 msgr2=0x55653c8d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 shutdown_connections 2024-09-13T23:43:26.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/4279797753 wait complete. 2024-09-13T23:43:26.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/3567060306 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653ccd0000 msgr2=0x55653ccd6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 --2- 172.21.15.167:0/3567060306 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653ccd0000 0x55653ccd6000 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x55653cbd1cb0 tx=0x55653cb2d7a0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 -- 172.21.15.167:0/3567060306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 msgr2=0x55653cab9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.059+0000 7f26109d2f40 1 --2- 172.21.15.167:0/3567060306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 0x55653cab9180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55653cc501b0 tx=0x55653cc52300 comp rx=0 tx=0).stop 2024-09-13T23:43:26.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 -- 172.21.15.167:0/3567060306 shutdown_connections 2024-09-13T23:43:26.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 --2- 172.21.15.167:0/3567060306 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653ccd0000 0x55653ccd6000 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 --2- 172.21.15.167:0/3567060306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653cc42c00 0x55653cab9180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 -- 172.21.15.167:0/3567060306 >> 172.21.15.167:0/3567060306 conn(0x55653cc43c00 msgr2=0x55653c8d2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 -- 172.21.15.167:0/3567060306 shutdown_connections 2024-09-13T23:43:26.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 -- 172.21.15.167:0/3567060306 wait complete. 2024-09-13T23:43:26.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 -- 172.21.15.167:0/888115354 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653cb8c000 msgr2=0x55653cab7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 --2- 172.21.15.167:0/888115354 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653cb8c000 0x55653cab7080 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55653cb667b0 tx=0x55653cb2c360 comp rx=0 tx=0).stop 2024-09-13T23:43:26.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 -- 172.21.15.167:0/888115354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 msgr2=0x55653cab6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 --2- 172.21.15.167:0/888115354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 0x55653cab6580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x55653d6420f0 tx=0x55653c8d5a40 comp rx=0 tx=0).stop 2024-09-13T23:43:26.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 -- 172.21.15.167:0/888115354 shutdown_connections 2024-09-13T23:43:26.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 --2- 172.21.15.167:0/888115354 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55653cb8c000 0x55653cab7080 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 --2- 172.21.15.167:0/888115354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55653c7d1000 0x55653cab6580 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 -- 172.21.15.167:0/888115354 >> 172.21.15.167:0/888115354 conn(0x55653c7d0c00 msgr2=0x55653c8d1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 -- 172.21.15.167:0/888115354 shutdown_connections 2024-09-13T23:43:26.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.063+0000 7f26109d2f40 1 -- 172.21.15.167:0/888115354 wait complete. 2024-09-13T23:43:26.076 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T23:43:26.076 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-09-13T23:43:26.077 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-09-13T23:43:26.077 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid foo --display-name Foo --email foo@foo.com --access-key 9te6NH5mcdcq0Tc5i8i1 --secret Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu --max-buckets 4 2024-09-13T23:43:26.127 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:26.127 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:26.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.119+0000 7fb29b6e4f40 1 Processor -- start 2024-09-13T23:43:26.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.119+0000 7fb29b6e4f40 1 -- start start 2024-09-13T23:43:26.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.119+0000 7fb29b6e4f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5625714b5400 0x56257179a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.119+0000 7fb29b6e4f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562571579500 con 0x5625714b5000 2024-09-13T23:43:26.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.119+0000 7fb29b6e4f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x562571542a00 con 0x5625714b5400 2024-09-13T23:43:26.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.119+0000 7fb29969c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5625714b5400 0x56257179a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.119+0000 7fb29969c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5625714b5400 0x56257179a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43386/0 (socket says 172.21.15.167:43386) 2024-09-13T23:43:26.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.119+0000 7fb299e9d700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5625714b5000 0x5625715b4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:40368/0 (socket says 172.21.15.167:40368) 2024-09-13T23:43:26.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.119+0000 7fb29969c700 1 -- 172.21.15.167:0/2807944758 learned_addr learned my addr 172.21.15.167:0/2807944758 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.119+0000 7fb29969c700 1 -- 172.21.15.167:0/2807944758 >> v1:172.21.15.12:6789/0 conn(0x5625714b5000 legacy=0x5625715b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.119+0000 7fb29969c700 1 -- 172.21.15.167:0/2807944758 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625714c7d40 con 0x5625714b5400 2024-09-13T23:43:26.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29969c700 1 --2- 172.21.15.167:0/2807944758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5400 0x56257179a580 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x562571795920 tx=0x5625715b9920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0d0df7d5ed6511 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb298e9b700 1 -- 172.21.15.167:0/2807944758 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625715594a0 con 0x5625714b5400 2024-09-13T23:43:26.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb298e9b700 1 -- 172.21.15.167:0/2807944758 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562571579800 con 0x5625714b5400 2024-09-13T23:43:26.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb298e9b700 1 -- 172.21.15.167:0/2807944758 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562571559600 con 0x5625714b5400 2024-09-13T23:43:26.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2807944758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5400 msgr2=0x56257179a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2807944758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5400 0x56257179a580 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x562571795920 tx=0x5625715b9920 comp rx=0 tx=0).stop 2024-09-13T23:43:26.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2807944758 shutdown_connections 2024-09-13T23:43:26.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2807944758 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5400 0x56257179a580 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2807944758 >> 172.21.15.167:0/2807944758 conn(0x5625714b4c00 msgr2=0x5625715b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2807944758 shutdown_connections 2024-09-13T23:43:26.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2807944758 wait complete. 2024-09-13T23:43:26.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29b6e4f40 1 Processor -- start 2024-09-13T23:43:26.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29b6e4f40 1 -- start start 2024-09-13T23:43:26.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29b6e4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 0x56257179a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29b6e4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562571542500 con 0x5625714b5000 2024-09-13T23:43:26.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb299e9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 0x56257179a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb299e9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 0x56257179a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43402/0 (socket says 172.21.15.167:43402) 2024-09-13T23:43:26.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb299e9d700 1 -- 172.21.15.167:0/1041012994 learned_addr learned my addr 172.21.15.167:0/1041012994 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb299e9d700 1 -- 172.21.15.167:0/1041012994 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625714c7ba0 con 0x5625714b5000 2024-09-13T23:43:26.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb299e9d700 1 --2- 172.21.15.167:0/1041012994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 0x56257179a580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x562571611e90 tx=0x5625715b9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6e596f4faba033 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.123+0000 7fb29869a700 1 -- 172.21.15.167:0/1041012994 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625715594a0 con 0x5625714b5000 2024-09-13T23:43:26.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29869a700 1 -- 172.21.15.167:0/1041012994 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562571579b00 con 0x5625714b5000 2024-09-13T23:43:26.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29869a700 1 -- 172.21.15.167:0/1041012994 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562571559600 con 0x5625714b5000 2024-09-13T23:43:26.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1041012994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 msgr2=0x56257179a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/1041012994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 0x56257179a580 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x562571611e90 tx=0x5625715b9b60 comp rx=0 tx=0).stop 2024-09-13T23:43:26.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1041012994 shutdown_connections 2024-09-13T23:43:26.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/1041012994 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 0x56257179a580 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1041012994 >> 172.21.15.167:0/1041012994 conn(0x5625714b4c00 msgr2=0x5625715b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1041012994 shutdown_connections 2024-09-13T23:43:26.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1041012994 wait complete. 2024-09-13T23:43:26.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 Processor -- start 2024-09-13T23:43:26.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 -- start start 2024-09-13T23:43:26.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 0x56257179a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625715423c0 con 0x5625714b5000 2024-09-13T23:43:26.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb299e9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 0x56257179a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb299e9d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 0x56257179a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43408/0 (socket says 172.21.15.167:43408) 2024-09-13T23:43:26.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb299e9d700 1 -- 172.21.15.167:0/557733517 learned_addr learned my addr 172.21.15.167:0/557733517 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb299e9d700 1 -- 172.21.15.167:0/557733517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625714c7a00 con 0x5625714b5000 2024-09-13T23:43:26.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb299e9d700 1 --2- 172.21.15.167:0/557733517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 0x56257179a580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x56257231a0f0 tx=0x5625715b9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb296e97700 1 -- 172.21.15.167:0/557733517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625715598c0 con 0x5625714b5000 2024-09-13T23:43:26.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb296e97700 1 -- 172.21.15.167:0/557733517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562571579c80 con 0x5625714b5000 2024-09-13T23:43:26.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/557733517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5625714c7860 con 0x5625714b5000 2024-09-13T23:43:26.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb296e97700 1 -- 172.21.15.167:0/557733517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625715594a0 con 0x5625714b5000 2024-09-13T23:43:26.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.127+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/557733517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625714c76c0 con 0x5625714b5000 2024-09-13T23:43:26.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb29b6e4f40 1 Processor -- start 2024-09-13T23:43:26.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb29b6e4f40 1 -- start start 2024-09-13T23:43:26.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb296e97700 1 -- 172.21.15.167:0/557733517 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5625715aeb00 con 0x5625714b5000 2024-09-13T23:43:26.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb29b6e4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 0x56257179ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb29b6e4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625714da640 con 0x5625714b5800 2024-09-13T23:43:26.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb296e97700 1 --2- 172.21.15.167:0/557733517 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562571852000 0x56257179b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb296e97700 1 -- 172.21.15.167:0/557733517 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56257151b180 con 0x5625714b5000 2024-09-13T23:43:26.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb299e9d700 1 --2- 172.21.15.167:0/557733517 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562571852000 0x56257179b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb29a69e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 0x56257179ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb29a69e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 0x56257179ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43422/0 (socket says 172.21.15.167:43422) 2024-09-13T23:43:26.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb29a69e700 1 -- 172.21.15.167:0/72429108 learned_addr learned my addr 172.21.15.167:0/72429108 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb299e9d700 1 --2- 172.21.15.167:0/557733517 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562571852000 0x56257179b080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x562571832570 tx=0x56257180c240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb29a69e700 1 -- 172.21.15.167:0/72429108 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625714c6000 con 0x5625714b5800 2024-09-13T23:43:26.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb29a69e700 1 --2- 172.21.15.167:0/72429108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 0x56257179ab00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5625716117a0 tx=0x56257180d0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115e1fbcaf72f574 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb294692700 1 -- 172.21.15.167:0/72429108 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625715598c0 con 0x5625714b5800 2024-09-13T23:43:26.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb294692700 1 -- 172.21.15.167:0/72429108 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56257186a000 con 0x5625714b5800 2024-09-13T23:43:26.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.131+0000 7fb294692700 1 -- 172.21.15.167:0/72429108 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625715594a0 con 0x5625714b5800 2024-09-13T23:43:26.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/72429108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 msgr2=0x56257179ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/72429108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 0x56257179ab00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5625716117a0 tx=0x56257180d0e0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/72429108 shutdown_connections 2024-09-13T23:43:26.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/72429108 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 0x56257179ab00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/72429108 >> 172.21.15.167:0/72429108 conn(0x5625714b5400 msgr2=0x5625715b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/72429108 shutdown_connections 2024-09-13T23:43:26.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/72429108 wait complete. 2024-09-13T23:43:26.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 Processor -- start 2024-09-13T23:43:26.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 -- start start 2024-09-13T23:43:26.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 0x56257179ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625714da8c0 con 0x5625714b5800 2024-09-13T23:43:26.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29a69e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 0x56257179ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29a69e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 0x56257179ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43436/0 (socket says 172.21.15.167:43436) 2024-09-13T23:43:26.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 learned_addr learned my addr 172.21.15.167:0/2094716154 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625714c6ea0 con 0x5625714b5800 2024-09-13T23:43:26.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29a69e700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 0x56257179ab00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x56257231a2a0 tx=0x56257180cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb292e8f700 1 -- 172.21.15.167:0/2094716154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562571559b80 con 0x5625714b5800 2024-09-13T23:43:26.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb292e8f700 1 -- 172.21.15.167:0/2094716154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56257186a180 con 0x5625714b5800 2024-09-13T23:43:26.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5625714c6820 con 0x5625714b5800 2024-09-13T23:43:26.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb292e8f700 1 -- 172.21.15.167:0/2094716154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625715598c0 con 0x5625714b5800 2024-09-13T23:43:26.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.135+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625714c69c0 con 0x5625714b5800 2024-09-13T23:43:26.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb292e8f700 1 -- 172.21.15.167:0/2094716154 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5625715af080 con 0x5625714b5800 2024-09-13T23:43:26.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb292e8f700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562571852c00 0x56257179b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb292e8f700 1 -- 172.21.15.167:0/2094716154 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56257151b340 con 0x5625714b5800 2024-09-13T23:43:26.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29969c700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562571852c00 0x56257179b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29969c700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562571852c00 0x56257179b600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x562571795920 tx=0x5625715b9800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562571853000 0x56257179bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853400 con 0x562571853000 2024-09-13T23:43:26.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29a69e700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562571853000 0x56257179bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29a69e700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562571853000 0x56257179bb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29b6e4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29b6e4f40 20 realm 2024-09-13T23:43:26.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853000 2024-09-13T23:43:26.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29b6e4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29b6e4f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:26.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.139+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853c00 con 0x562571853000 2024-09-13T23:43:26.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29b6e4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562571853400 0x56257179c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853400 2024-09-13T23:43:26.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb299e9d700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562571853400 0x56257179c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb299e9d700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562571853400 0x56257179c100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29b6e4f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:26.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853c00 con 0x562571853400 2024-09-13T23:43:26.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29b6e4f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:26.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29b6e4f40 20 searching for the correct realm 2024-09-13T23:43:26.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56257191a000 0x56257179c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191a400 con 0x56257191a000 2024-09-13T23:43:26.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29969c700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56257191a000 0x56257179c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.143+0000 7fb29969c700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56257191a000 0x56257179c680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191a400 con 0x562571853400 2024-09-13T23:43:26.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb299e9d700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56257191a400 0x56257179cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a400 2024-09-13T23:43:26.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb29a69e700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56257191a400 0x56257179cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb29a69e700 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56257191a400 0x56257179cc00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853000 2024-09-13T23:43:26.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853000 2024-09-13T23:43:26.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.147+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a000 2024-09-13T23:43:26.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.151+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.151+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853000 2024-09-13T23:43:26.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.151+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.151+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853400 2024-09-13T23:43:26.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.151+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.151+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853400 2024-09-13T23:43:26.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.151+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.151+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853000 2024-09-13T23:43:26.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.151+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.151+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a400 2024-09-13T23:43:26.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.151+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.151+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853000 2024-09-13T23:43:26.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.155+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.155+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a400 2024-09-13T23:43:26.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.155+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.155+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853400 2024-09-13T23:43:26.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.155+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.155+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a000 2024-09-13T23:43:26.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.155+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.155+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853000 2024-09-13T23:43:26.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.155+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.155+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853400 2024-09-13T23:43:26.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.155+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.155+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a000 2024-09-13T23:43:26.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853000 2024-09-13T23:43:26.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853000 2024-09-13T23:43:26.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853000 2024-09-13T23:43:26.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a000 2024-09-13T23:43:26.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a400 2024-09-13T23:43:26.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853000 2024-09-13T23:43:26.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.159+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a400 2024-09-13T23:43:26.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.163+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.163+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a000 2024-09-13T23:43:26.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.163+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.163+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a400 2024-09-13T23:43:26.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.163+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.163+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853400 2024-09-13T23:43:26.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.163+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.163+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853400 2024-09-13T23:43:26.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.163+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.163+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a000 2024-09-13T23:43:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.163+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.163+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853400 2024-09-13T23:43:26.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853400 2024-09-13T23:43:26.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:26.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:26.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:26.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:26.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191a800 con 0x562571853000 2024-09-13T23:43:26.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191ac00 con 0x562571853400 2024-09-13T23:43:26.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:26.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191b000 con 0x562571853400 2024-09-13T23:43:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 zone default found 2024-09-13T23:43:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 4 Realm: () 2024-09-13T23:43:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 zonegroup default 2024-09-13T23:43:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.167+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191b400 con 0x56257191a400 2024-09-13T23:43:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29b6e4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191b800 con 0x56257191a000 2024-09-13T23:43:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29b6e4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191bc00 con 0x56257191a000 2024-09-13T23:43:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29b6e4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29b6e4f40 20 started sync module instance, tier type = 2024-09-13T23:43:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29b6e4f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:26.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.171+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191b800 con 0x562571853400 2024-09-13T23:43:26.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.175+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.175+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94718817540864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191bc00 con 0x562571853400 2024-09-13T23:43:26.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.175+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191ac00 con 0x562571853000 2024-09-13T23:43:26.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.175+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94718817540864] v33'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.179+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.179+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94718817542016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191b000 con 0x562571853000 2024-09-13T23:43:26.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.179+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191b800 con 0x562571853400 2024-09-13T23:43:26.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.179+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94718817542016] v33'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.179+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.179+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94718817543168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853c00 con 0x562571853400 2024-09-13T23:43:26.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.179+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a400 2024-09-13T23:43:26.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.183+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.183+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94718817544320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191a800 con 0x56257191a400 2024-09-13T23:43:26.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.183+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191b400 con 0x562571853400 2024-09-13T23:43:26.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.183+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94718817543168] v33'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.187+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.187+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94718817545472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191ac00 con 0x562571853400 2024-09-13T23:43:26.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.187+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191b000 con 0x56257191a000 2024-09-13T23:43:26.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.187+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94718817544320] v33'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.187+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.187+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94718817546624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257192a000 con 0x56257191a000 2024-09-13T23:43:26.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.187+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257192a400 con 0x56257191a000 2024-09-13T23:43:26.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.187+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94718817545472] v33'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94718817546624] v33'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94718817547776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257192a800 con 0x56257191a000 2024-09-13T23:43:26.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257192ac00 con 0x56257191a400 2024-09-13T23:43:26.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'24 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94718817548928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257192b000 con 0x56257191a400 2024-09-13T23:43:26.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29b6e4f40 20 add_watcher() i=0 2024-09-13T23:43:26.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29b6e4f40 20 add_watcher() i=1 2024-09-13T23:43:26.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29b6e4f40 20 add_watcher() i=2 2024-09-13T23:43:26.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29b6e4f40 20 add_watcher() i=3 2024-09-13T23:43:26.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29b6e4f40 20 add_watcher() i=4 2024-09-13T23:43:26.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29b6e4f40 20 add_watcher() i=5 2024-09-13T23:43:26.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94718817547776] v33'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.191+0000 7fb29b6e4f40 20 add_watcher() i=6 2024-09-13T23:43:26.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.195+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94718817548928] v33'25 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.195+0000 7fb29b6e4f40 20 add_watcher() i=7 2024-09-13T23:43:26.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.195+0000 7fb29b6e4f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:26.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.195+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257192b400 con 0x562571853000 2024-09-13T23:43:26.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.195+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.195+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94718819221504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257192b800 con 0x562571853000 2024-09-13T23:43:26.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.195+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94718819221504] v33'50 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.195+0000 7fb280e6b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:26.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.195+0000 7fb29b6e4f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:26.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.195+0000 7fb29b6e4f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:26.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.195+0000 7fb29b6e4f40 1 Processor -- start 2024-09-13T23:43:26.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.195+0000 7fb29b6e4f40 1 -- start start 2024-09-13T23:43:26.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29b6e4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 0x56257179d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29b6e4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625715423c0 con 0x56257192ac00 2024-09-13T23:43:26.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29969c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 0x56257179d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29969c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 0x56257179d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43440/0 (socket says 172.21.15.167:43440) 2024-09-13T23:43:26.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29969c700 1 -- 172.21.15.167:0/2397087847 learned_addr learned my addr 172.21.15.167:0/2397087847 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29969c700 1 -- 172.21.15.167:0/2397087847 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625714c6b60 con 0x56257192ac00 2024-09-13T23:43:26.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29969c700 1 --2- 172.21.15.167:0/2397087847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 0x56257179d180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x5625718b5620 tx=0x5625715b9860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b312310d925edb46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb28066a700 1 -- 172.21.15.167:0/2397087847 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562571559760 con 0x56257192ac00 2024-09-13T23:43:26.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb28066a700 1 -- 172.21.15.167:0/2397087847 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56257186a600 con 0x56257192ac00 2024-09-13T23:43:26.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb28066a700 1 -- 172.21.15.167:0/2397087847 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562571944000 con 0x56257192ac00 2024-09-13T23:43:26.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2397087847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 msgr2=0x56257179d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2397087847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 0x56257179d180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x5625718b5620 tx=0x5625715b9860 comp rx=0 tx=0).stop 2024-09-13T23:43:26.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2397087847 shutdown_connections 2024-09-13T23:43:26.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2397087847 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 0x56257179d180 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2397087847 >> 172.21.15.167:0/2397087847 conn(0x56257192bc00 msgr2=0x5625715b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2397087847 shutdown_connections 2024-09-13T23:43:26.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2397087847 wait complete. 2024-09-13T23:43:26.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29b6e4f40 1 Processor -- start 2024-09-13T23:43:26.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29b6e4f40 1 -- start start 2024-09-13T23:43:26.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.199+0000 7fb29b6e4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 0x56257179d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb29b6e4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562571542500 con 0x56257192ac00 2024-09-13T23:43:26.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb29969c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 0x56257179d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb29969c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 0x56257179d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43450/0 (socket says 172.21.15.167:43450) 2024-09-13T23:43:26.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb29969c700 1 -- 172.21.15.167:0/1335161977 learned_addr learned my addr 172.21.15.167:0/1335161977 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb29969c700 1 -- 172.21.15.167:0/1335161977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625714c6d00 con 0x56257192ac00 2024-09-13T23:43:26.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb29969c700 1 --2- 172.21.15.167:0/1335161977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 0x56257179d180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x562571934810 tx=0x56257193a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb27ee67700 1 -- 172.21.15.167:0/1335161977 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562571944160 con 0x56257192ac00 2024-09-13T23:43:26.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb27ee67700 1 -- 172.21.15.167:0/1335161977 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56257186a300 con 0x56257192ac00 2024-09-13T23:43:26.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb27ee67700 1 -- 172.21.15.167:0/1335161977 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5625719442c0 con 0x56257192ac00 2024-09-13T23:43:26.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1335161977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5625714c6340 con 0x56257192ac00 2024-09-13T23:43:26.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1335161977 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5625714c7d40 con 0x56257192ac00 2024-09-13T23:43:26.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb27ee67700 1 -- 172.21.15.167:0/1335161977 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5625715afb80 con 0x56257192ac00 2024-09-13T23:43:26.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb27ee67700 1 --2- 172.21.15.167:0/1335161977 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5625719b8000 0x5625719be000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb27ee67700 1 -- 172.21.15.167:0/1335161977 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56257151b500 con 0x56257192ac00 2024-09-13T23:43:26.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb29b6e4f40 5 note: GC not initialized 2024-09-13T23:43:26.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.203+0000 7fb29a69e700 1 --2- 172.21.15.167:0/1335161977 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5625719b8000 0x5625719be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29a69e700 1 --2- 172.21.15.167:0/1335161977 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5625719b8000 0x5625719be000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5625718b5c80 tx=0x56257180d7a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb27c662700 20 reqs_thread_entry: start 2024-09-13T23:43:26.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:26.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:26.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb27a65e700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:56 2024 2024-09-13T23:43:26.218 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:26.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb279e3c700 20 reqs_thread_entry: start 2024-09-13T23:43:26.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257192a000 con 0x562571853000 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257192a400 con 0x562571853000 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 foo@foo.com [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 197+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x0 2024-09-13T23:43:26.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-09-13T23:43:26.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : miss 2024-09-13T23:43:26.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191a800 con 0x562571853000 2024-09-13T23:43:26.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(64 9te6NH5mcdcq0Tc5i8i1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x0 2024-09-13T23:43:26.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 10 adding default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-09-13T23:43:26.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-09-13T23:43:26.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-09-13T23:43:26.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.207+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191b000 con 0x562571853000 2024-09-13T23:43:26.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.211+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(65 foo [delete,create,call,writefull 0~357] v33'1 uv1 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.211+0000 7fb29b6e4f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:26.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.211+0000 7fb29b6e4f40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:26.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.211+0000 7fb29b6e4f40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:26.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.211+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify cookie 94718819223808 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x562571853400 2024-09-13T23:43:26.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.211+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94718817545472 notify 141733920768 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x56257180a820 con 0x562571853400 2024-09-13T23:43:26.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.211+0000 7fb294692700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920768 cookie 94718817545472 notifier 4294 bl.length()=533 2024-09-13T23:43:26.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.211+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.4 [notify cookie 94718819223808 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.211+0000 7fb294692700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:26.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.211+0000 7fb294692700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:26.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.211+0000 7fb294692700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191bc00 con 0x562571853400 2024-09-13T23:43:26.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.215+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(67 notify.4 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.215+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94718819223808 notify 141733920768 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56257180a9c0 con 0x562571853400 2024-09-13T23:43:26.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.215+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5625719b8400 con 0x562571853000 2024-09-13T23:43:26.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.215+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(68 foo@foo.com [delete,create,writefull 0~7] v33'1 uv1 ondisk = 0) v8 ==== 239+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.215+0000 7fb29b6e4f40 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-09-13T23:43:26.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.215+0000 7fb29b6e4f40 10 moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-09-13T23:43:26.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.215+0000 7fb29b6e4f40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-09-13T23:43:26.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.215+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [notify cookie 94718819223808 in=181b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5625719b8800 con 0x56257191a400 2024-09-13T23:43:26.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== watch-notify(notify (1) cookie 94718817544320 notify 141733920768 ret 0) v3 ==== 211+0+0 (crc 0 0 0) 0x5625718b8820 con 0x56257191a400 2024-09-13T23:43:26.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb294692700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920768 cookie 94718817544320 notifier 4294 bl.length()=169 2024-09-13T23:43:26.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb294692700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-09-13T23:43:26.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb294692700 10 rgw watcher librados: moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-09-13T23:43:26.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(69 notify.3 [notify cookie 94718819223808 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb294692700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191b800 con 0x56257191a400 2024-09-13T23:43:26.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== watch-notify(notify_complete (2) cookie 94718819223808 notify 141733920768 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5625718b89c0 con 0x56257191a400 2024-09-13T23:43:26.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(70 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:71 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5625719b8c00 con 0x562571853000 2024-09-13T23:43:26.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(71 9te6NH5mcdcq0Tc5i8i1 [delete,create,writefull 0~7] v33'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb29b6e4f40 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-09-13T23:43:26.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb29b6e4f40 10 moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-09-13T23:43:26.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb29b6e4f40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-09-13T23:43:26.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [notify cookie 94718819223808 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5625719b9000 con 0x56257191a400 2024-09-13T23:43:26.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== watch-notify(notify (1) cookie 94718817544320 notify 141733920769 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x5625718b89c0 con 0x56257191a400 2024-09-13T23:43:26.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb294692700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920769 cookie 94718817544320 notifier 4294 bl.length()=177 2024-09-13T23:43:26.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb294692700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-09-13T23:43:26.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb294692700 10 rgw watcher librados: moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-09-13T23:43:26.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(72 notify.3 [notify cookie 94718819223808 out=8b] v0'0 uv24 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.219+0000 7fb294692700 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257192b800 con 0x56257191a400 2024-09-13T23:43:26.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.223+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 18 ==== watch-notify(notify_complete (2) cookie 94718819223808 notify 141733920769 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5625718b8b60 con 0x56257191a400 2024-09-13T23:43:26.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.223+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 19 ==== osd_op_reply(73 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.232 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:26.232 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:26.232 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:26.232 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:26.232 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:26.232 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:26.232 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:26.232 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:26.233 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:26.233 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:26.233 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:26.233 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:26.233 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:26.233 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:26.233 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:26.233 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:26.233 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:26.233 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:26.233 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:26.234 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:26.235 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:26.235 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:26.235 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:26.235 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:26.235 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:26.235 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:26.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.223+0000 7fb280e6b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:26.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.223+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:74 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94718819221504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5625719b9400 con 0x562571853000 2024-09-13T23:43:26.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.227+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(74 data_loggenerations_metadata [watch unwatch cookie 94718819221504] v33'51 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.227+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94718817540864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5625719b9800 con 0x562571853400 2024-09-13T23:43:26.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.227+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(75 notify.0 [watch unwatch cookie 94718817540864] v33'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.227+0000 7fb29b6e4f40 20 remove_watcher() i=0 2024-09-13T23:43:26.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.227+0000 7fb29b6e4f40 2 removed watcher, disabling cache 2024-09-13T23:43:26.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.227+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:76 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94718817542016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5625719b9c00 con 0x562571853000 2024-09-13T23:43:26.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.231+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(76 notify.1 [watch unwatch cookie 94718817542016] v33'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190eb40 con 0x562571853000 2024-09-13T23:43:26.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.231+0000 7fb29b6e4f40 20 remove_watcher() i=1 2024-09-13T23:43:26.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.231+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:77 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94718817543168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853c00 con 0x562571853400 2024-09-13T23:43:26.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.235+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(77 notify.2 [watch unwatch cookie 94718817543168] v33'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.235+0000 7fb29b6e4f40 20 remove_watcher() i=2 2024-09-13T23:43:26.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.235+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94718817544320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x562571853800 con 0x56257191a400 2024-09-13T23:43:26.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.235+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 20 ==== osd_op_reply(78 notify.3 [watch unwatch cookie 94718817544320] v33'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.235+0000 7fb29b6e4f40 20 remove_watcher() i=3 2024-09-13T23:43:26.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.235+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:79 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94718817545472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191b400 con 0x562571853400 2024-09-13T23:43:26.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.239+0000 7fb299e9d700 1 -- 172.21.15.167:0/2094716154 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(79 notify.4 [watch unwatch cookie 94718817545472] v33'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f200 con 0x562571853400 2024-09-13T23:43:26.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.239+0000 7fb29b6e4f40 20 remove_watcher() i=4 2024-09-13T23:43:26.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.239+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:80 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94718817546624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56257191ac00 con 0x56257191a000 2024-09-13T23:43:26.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.239+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(80 notify.5 [watch unwatch cookie 94718817546624] v33'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.243+0000 7fb29b6e4f40 20 remove_watcher() i=5 2024-09-13T23:43:26.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.243+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:81 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94718817547776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5625719b8800 con 0x56257191a000 2024-09-13T23:43:26.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.243+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(81 notify.6 [watch unwatch cookie 94718817547776] v33'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190f440 con 0x56257191a000 2024-09-13T23:43:26.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.243+0000 7fb29b6e4f40 20 remove_watcher() i=6 2024-09-13T23:43:26.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.243+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:82 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94718817548928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5625719b8c00 con 0x56257191a400 2024-09-13T23:43:26.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29a69e700 1 -- 172.21.15.167:0/2094716154 <== osd.2 v2:172.21.15.167:6808/1180369139 21 ==== osd_op_reply(82 notify.7 [watch unwatch cookie 94718817548928] v33'26 uv24 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56257190eb40 con 0x56257191a400 2024-09-13T23:43:26.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 20 remove_watcher() i=7 2024-09-13T23:43:26.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562571853400 msgr2=0x56257179c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562571853400 0x56257179c100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562571853000 msgr2=0x56257179bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562571853000 0x56257179bb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56257191a400 msgr2=0x56257179cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56257191a400 0x56257179cc00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56257191a000 msgr2=0x56257179c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56257191a000 0x56257179c680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562571852c00 msgr2=0x56257179b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562571852c00 0x56257179b600 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x562571795920 tx=0x5625715b9800 comp rx=0 tx=0).stop 2024-09-13T23:43:26.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 msgr2=0x56257179ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 0x56257179ab00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x56257231a2a0 tx=0x56257180cd20 comp rx=0 tx=0).stop 2024-09-13T23:43:26.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29969c700 1 -- 172.21.15.167:0/2094716154 reap_dead start 2024-09-13T23:43:26.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 shutdown_connections 2024-09-13T23:43:26.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/2094716154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5800 0x56257179ab00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 >> 172.21.15.167:0/2094716154 conn(0x5625714b5400 msgr2=0x5625715b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 shutdown_connections 2024-09-13T23:43:26.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/2094716154 wait complete. 2024-09-13T23:43:26.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1335161977 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5625719b8000 msgr2=0x5625719be000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/1335161977 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5625719b8000 0x5625719be000 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5625718b5c80 tx=0x56257180d7a0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1335161977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 msgr2=0x56257179d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/1335161977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 0x56257179d180 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x562571934810 tx=0x56257193a300 comp rx=0 tx=0).stop 2024-09-13T23:43:26.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1335161977 shutdown_connections 2024-09-13T23:43:26.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/1335161977 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5625719b8000 0x5625719be000 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/1335161977 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56257192ac00 0x56257179d180 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1335161977 >> 172.21.15.167:0/1335161977 conn(0x56257192bc00 msgr2=0x5625715b6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1335161977 shutdown_connections 2024-09-13T23:43:26.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/1335161977 wait complete. 2024-09-13T23:43:26.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/557733517 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562571852000 msgr2=0x56257179b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/557733517 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562571852000 0x56257179b080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x562571832570 tx=0x56257180c240 comp rx=0 tx=0).stop 2024-09-13T23:43:26.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/557733517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 msgr2=0x56257179a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/557733517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 0x56257179a580 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x56257231a0f0 tx=0x5625715b9a40 comp rx=0 tx=0).stop 2024-09-13T23:43:26.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/557733517 shutdown_connections 2024-09-13T23:43:26.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/557733517 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562571852000 0x56257179b080 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 --2- 172.21.15.167:0/557733517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5625714b5000 0x56257179a580 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/557733517 >> 172.21.15.167:0/557733517 conn(0x5625714b4c00 msgr2=0x5625715b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/557733517 shutdown_connections 2024-09-13T23:43:26.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.247+0000 7fb29b6e4f40 1 -- 172.21.15.167:0/557733517 wait complete. 2024-09-13T23:43:26.264 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:26.264 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-09-13T23:43:26.264 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-09-13T23:43:26.264 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --email foo@foo.com 2024-09-13T23:43:26.305 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:26.305 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:26.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.295+0000 7f60f2b0bf40 1 Processor -- start 2024-09-13T23:43:26.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.295+0000 7f60f2b0bf40 1 -- start start 2024-09-13T23:43:26.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.295+0000 7f60f2b0bf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x560c2eb47000 0x560c2ee2c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.295+0000 7f60f2b0bf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x560c2ec0b500 con 0x560c2eb47400 2024-09-13T23:43:26.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.295+0000 7f60f2b0bf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x560c2ebd4a00 con 0x560c2eb47000 2024-09-13T23:43:26.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f12c4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x560c2eb47000 0x560c2ee2c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f12c4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x560c2eb47000 0x560c2ee2c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43458/0 (socket says 172.21.15.167:43458) 2024-09-13T23:43:26.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f12c4700 1 -- 172.21.15.167:0/3337185819 learned_addr learned my addr 172.21.15.167:0/3337185819 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f0ac3700 1 --1- 172.21.15.167:0/3337185819 >> v1:172.21.15.12:6789/0 conn(0x560c2eb47400 0x560c2ec46800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:40372/0 (socket says 172.21.15.167:40372) 2024-09-13T23:43:26.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f12c4700 1 -- 172.21.15.167:0/3337185819 >> v1:172.21.15.12:6789/0 conn(0x560c2eb47400 legacy=0x560c2ec46800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f12c4700 1 -- 172.21.15.167:0/3337185819 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c2eb59d40 con 0x560c2eb47000 2024-09-13T23:43:26.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f12c4700 1 --2- 172.21.15.167:0/3337185819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47000 0x560c2ee2c580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x560c2ee27a70 tx=0x560c2ec4b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fc648d2a6be2e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f02c2700 1 -- 172.21.15.167:0/3337185819 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2ebeb4a0 con 0x560c2eb47000 2024-09-13T23:43:26.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f02c2700 1 -- 172.21.15.167:0/3337185819 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c2ec0b800 con 0x560c2eb47000 2024-09-13T23:43:26.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f02c2700 1 -- 172.21.15.167:0/3337185819 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2ebeb600 con 0x560c2eb47000 2024-09-13T23:43:26.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3337185819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47000 msgr2=0x560c2ee2c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/3337185819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47000 0x560c2ee2c580 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x560c2ee27a70 tx=0x560c2ec4b920 comp rx=0 tx=0).stop 2024-09-13T23:43:26.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3337185819 shutdown_connections 2024-09-13T23:43:26.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/3337185819 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47000 0x560c2ee2c580 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3337185819 >> 172.21.15.167:0/3337185819 conn(0x560c2eb46c00 msgr2=0x560c2ec46000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3337185819 shutdown_connections 2024-09-13T23:43:26.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3337185819 wait complete. 2024-09-13T23:43:26.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f2b0bf40 1 Processor -- start 2024-09-13T23:43:26.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f2b0bf40 1 -- start start 2024-09-13T23:43:26.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f2b0bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47000 0x560c2ee2c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f2b0bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2ebd4500 con 0x560c2eb47000 2024-09-13T23:43:26.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f12c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47000 0x560c2ee2c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f12c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47000 0x560c2ee2c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43462/0 (socket says 172.21.15.167:43462) 2024-09-13T23:43:26.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.299+0000 7f60f12c4700 1 -- 172.21.15.167:0/1408987668 learned_addr learned my addr 172.21.15.167:0/1408987668 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f12c4700 1 -- 172.21.15.167:0/1408987668 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c2eb59ba0 con 0x560c2eb47000 2024-09-13T23:43:26.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f12c4700 1 --2- 172.21.15.167:0/1408987668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47000 0x560c2ee2c580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x560c2eca21b0 tx=0x560c2ec4b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c32fdfdfeb82d10 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60efac1700 1 -- 172.21.15.167:0/1408987668 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2ebeb760 con 0x560c2eb47000 2024-09-13T23:43:26.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60efac1700 1 -- 172.21.15.167:0/1408987668 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c2ec0bb00 con 0x560c2eb47000 2024-09-13T23:43:26.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60efac1700 1 -- 172.21.15.167:0/1408987668 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2ebeb4a0 con 0x560c2eb47000 2024-09-13T23:43:26.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/1408987668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47000 msgr2=0x560c2ee2c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/1408987668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47000 0x560c2ee2c580 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x560c2eca21b0 tx=0x560c2ec4b800 comp rx=0 tx=0).stop 2024-09-13T23:43:26.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/1408987668 shutdown_connections 2024-09-13T23:43:26.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/1408987668 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47000 0x560c2ee2c580 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/1408987668 >> 172.21.15.167:0/1408987668 conn(0x560c2eb46c00 msgr2=0x560c2ec47000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/1408987668 shutdown_connections 2024-09-13T23:43:26.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/1408987668 wait complete. 2024-09-13T23:43:26.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f2b0bf40 1 Processor -- start 2024-09-13T23:43:26.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f2b0bf40 1 -- start start 2024-09-13T23:43:26.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f2b0bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47400 0x560c2ee2cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f2b0bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2ebd43c0 con 0x560c2eb47400 2024-09-13T23:43:26.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f12c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47400 0x560c2ee2cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f12c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47400 0x560c2ee2cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43470/0 (socket says 172.21.15.167:43470) 2024-09-13T23:43:26.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f12c4700 1 -- 172.21.15.167:0/578733959 learned_addr learned my addr 172.21.15.167:0/578733959 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f12c4700 1 -- 172.21.15.167:0/578733959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c2eb59a00 con 0x560c2eb47400 2024-09-13T23:43:26.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60f12c4700 1 --2- 172.21.15.167:0/578733959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47400 0x560c2ee2cb00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x560c2eca20f0 tx=0x560c2ec4b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.303+0000 7f60ee2be700 1 -- 172.21.15.167:0/578733959 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2ebeb600 con 0x560c2eb47400 2024-09-13T23:43:26.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60ee2be700 1 -- 172.21.15.167:0/578733959 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c2ec0bc80 con 0x560c2eb47400 2024-09-13T23:43:26.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/578733959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560c2eb59860 con 0x560c2eb47400 2024-09-13T23:43:26.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60ee2be700 1 -- 172.21.15.167:0/578733959 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2ebeb8c0 con 0x560c2eb47400 2024-09-13T23:43:26.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/578733959 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c2eb596c0 con 0x560c2eb47400 2024-09-13T23:43:26.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f2b0bf40 1 Processor -- start 2024-09-13T23:43:26.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f2b0bf40 1 -- start start 2024-09-13T23:43:26.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60ee2be700 1 -- 172.21.15.167:0/578733959 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x560c2ec40b00 con 0x560c2eb47400 2024-09-13T23:43:26.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f2b0bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 0x560c2ee2d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f2b0bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2eb6c640 con 0x560c2eb47c00 2024-09-13T23:43:26.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60ee2be700 1 --2- 172.21.15.167:0/578733959 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2eee8000 0x560c2ee2e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60ee2be700 1 -- 172.21.15.167:0/578733959 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x560c2ebad180 con 0x560c2eb47400 2024-09-13T23:43:26.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f1ac5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 0x560c2ee2d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f1ac5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 0x560c2ee2d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43472/0 (socket says 172.21.15.167:43472) 2024-09-13T23:43:26.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f1ac5700 1 -- 172.21.15.167:0/2719068420 learned_addr learned my addr 172.21.15.167:0/2719068420 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f12c4700 1 --2- 172.21.15.167:0/578733959 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2eee8000 0x560c2ee2e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f1ac5700 1 -- 172.21.15.167:0/2719068420 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c2eb58000 con 0x560c2eb47c00 2024-09-13T23:43:26.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f12c4700 1 --2- 172.21.15.167:0/578733959 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2eee8000 0x560c2ee2e100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x560c2ee6ae40 tx=0x560c2eeaab40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60f1ac5700 1 --2- 172.21.15.167:0/2719068420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 0x560c2ee2d080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x560c2ee6aa20 tx=0x560c2eeaae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef423f5d8f96f115 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60ebab9700 1 -- 172.21.15.167:0/2719068420 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2ebeb600 con 0x560c2eb47c00 2024-09-13T23:43:26.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60ebab9700 1 -- 172.21.15.167:0/2719068420 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c2eef6000 con 0x560c2eb47c00 2024-09-13T23:43:26.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.307+0000 7f60ebab9700 1 -- 172.21.15.167:0/2719068420 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2ebeb8c0 con 0x560c2eb47c00 2024-09-13T23:43:26.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2719068420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 msgr2=0x560c2ee2d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/2719068420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 0x560c2ee2d080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x560c2ee6aa20 tx=0x560c2eeaae40 comp rx=0 tx=0).stop 2024-09-13T23:43:26.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2719068420 shutdown_connections 2024-09-13T23:43:26.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/2719068420 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 0x560c2ee2d080 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2719068420 >> 172.21.15.167:0/2719068420 conn(0x560c2eb47800 msgr2=0x560c2ec47800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2719068420 shutdown_connections 2024-09-13T23:43:26.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2719068420 wait complete. 2024-09-13T23:43:26.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 Processor -- start 2024-09-13T23:43:26.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 -- start start 2024-09-13T23:43:26.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 0x560c2ee2d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2eb6c8c0 con 0x560c2eb47c00 2024-09-13T23:43:26.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f1ac5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 0x560c2ee2d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f1ac5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 0x560c2ee2d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43486/0 (socket says 172.21.15.167:43486) 2024-09-13T23:43:26.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 learned_addr learned my addr 172.21.15.167:0/3380482707 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c2eb58ea0 con 0x560c2eb47c00 2024-09-13T23:43:26.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f1ac5700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 0x560c2ee2d080 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x560c2ee6a7b0 tx=0x560c2ec4bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60ea2b6700 1 -- 172.21.15.167:0/3380482707 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2ebeb760 con 0x560c2eb47c00 2024-09-13T23:43:26.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60ea2b6700 1 -- 172.21.15.167:0/3380482707 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c2eef6180 con 0x560c2eb47c00 2024-09-13T23:43:26.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60ea2b6700 1 -- 172.21.15.167:0/3380482707 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2ebeb600 con 0x560c2eb47c00 2024-09-13T23:43:26.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560c2eb58820 con 0x560c2eb47c00 2024-09-13T23:43:26.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.311+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c2eb589c0 con 0x560c2eb47c00 2024-09-13T23:43:26.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60ea2b6700 1 -- 172.21.15.167:0/3380482707 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x560c2ec41080 con 0x560c2eb47c00 2024-09-13T23:43:26.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60ea2b6700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2eee8c00 0x560c2ee2e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60ea2b6700 1 -- 172.21.15.167:0/3380482707 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x560c2ebad340 con 0x560c2eb47c00 2024-09-13T23:43:26.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f0ac3700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2eee8c00 0x560c2ee2e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560c2eee9000 0x560c2ee2d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2eee9400 con 0x560c2eee9000 2024-09-13T23:43:26.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f1ac5700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560c2eee9000 0x560c2ee2d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f0ac3700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2eee8c00 0x560c2ee2e680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x560c2ee27f80 tx=0x560c2ef9a0c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f1ac5700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560c2eee9000 0x560c2ee2d600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 20 realm 2024-09-13T23:43:26.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2eee9800 con 0x560c2eee9000 2024-09-13T23:43:26.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:26.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2eee9c00 con 0x560c2eee9000 2024-09-13T23:43:26.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.315+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560c2eee9400 0x560c2ee2db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2eee9800 con 0x560c2eee9400 2024-09-13T23:43:26.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f12c4700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560c2eee9400 0x560c2ee2db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f12c4700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560c2eee9400 0x560c2ee2db80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f2b0bf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:26.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2eee9c00 con 0x560c2eee9400 2024-09-13T23:43:26.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f2b0bf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:26.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f2b0bf40 20 searching for the correct realm 2024-09-13T23:43:26.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560c2eee9800 0x560c2ee2ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2eee9c00 con 0x560c2eee9800 2024-09-13T23:43:26.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f0ac3700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560c2eee9800 0x560c2ee2ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f0ac3700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560c2eee9800 0x560c2ee2ec00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.319+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2eee9c00 con 0x560c2eee9400 2024-09-13T23:43:26.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f12c4700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560c2eee9c00 0x560c2ee2c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9c00 2024-09-13T23:43:26.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f1ac5700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560c2eee9c00 0x560c2ee2c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f1ac5700 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560c2eee9c00 0x560c2ee2c580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9000 2024-09-13T23:43:26.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9000 2024-09-13T23:43:26.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9800 2024-09-13T23:43:26.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9000 2024-09-13T23:43:26.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.323+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9400 2024-09-13T23:43:26.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.327+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.327+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9400 2024-09-13T23:43:26.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.327+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.327+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9000 2024-09-13T23:43:26.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.327+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.327+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9c00 2024-09-13T23:43:26.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.327+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.327+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9000 2024-09-13T23:43:26.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.327+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.327+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9c00 2024-09-13T23:43:26.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.327+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.327+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9400 2024-09-13T23:43:26.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9800 2024-09-13T23:43:26.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9000 2024-09-13T23:43:26.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9400 2024-09-13T23:43:26.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9800 2024-09-13T23:43:26.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9000 2024-09-13T23:43:26.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9000 2024-09-13T23:43:26.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9000 2024-09-13T23:43:26.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.331+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9800 2024-09-13T23:43:26.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9c00 2024-09-13T23:43:26.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9000 2024-09-13T23:43:26.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9c00 2024-09-13T23:43:26.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9800 2024-09-13T23:43:26.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9c00 2024-09-13T23:43:26.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9400 2024-09-13T23:43:26.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.335+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9400 2024-09-13T23:43:26.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9800 2024-09-13T23:43:26.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9400 2024-09-13T23:43:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9400 2024-09-13T23:43:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f2b0bf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f2b0bf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f2b0bf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f2b0bf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:26.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae800 con 0x560c2eee9000 2024-09-13T23:43:26.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f2b0bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.339+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaec00 con 0x560c2eee9400 2024-09-13T23:43:26.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:26.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaf000 con 0x560c2eee9400 2024-09-13T23:43:26.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:26.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 20 zone default found 2024-09-13T23:43:26.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 4 Realm: () 2024-09-13T23:43:26.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:26.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:26.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:26.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 20 zonegroup default 2024-09-13T23:43:26.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaf400 con 0x560c2eee9c00 2024-09-13T23:43:26.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaf800 con 0x560c2eee9800 2024-09-13T23:43:26.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efafc00 con 0x560c2eee9800 2024-09-13T23:43:26.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.343+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.347+0000 7f60f2b0bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.347+0000 7f60f2b0bf40 20 started sync module instance, tier type = 2024-09-13T23:43:26.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.347+0000 7f60f2b0bf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:26.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.347+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaf800 con 0x560c2eee9400 2024-09-13T23:43:26.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.347+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.347+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94610326174464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efafc00 con 0x560c2eee9400 2024-09-13T23:43:26.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.347+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae400 con 0x560c2eee9000 2024-09-13T23:43:26.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.351+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94610326174464] v33'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.351+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.351+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94610326175616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9000 2024-09-13T23:43:26.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.351+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaec00 con 0x560c2eee9400 2024-09-13T23:43:26.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.355+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94610326175616] v33'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.355+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.355+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94610326176768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaf000 con 0x560c2eee9400 2024-09-13T23:43:26.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.355+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae800 con 0x560c2eee9c00 2024-09-13T23:43:26.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.355+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.355+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94610326177920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaf400 con 0x560c2eee9c00 2024-09-13T23:43:26.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.355+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae400 con 0x560c2eee9400 2024-09-13T23:43:26.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.359+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94610326176768] v33'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.359+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94610326177920] v33'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.359+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.359+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94610326179072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae000 con 0x560c2eee9400 2024-09-13T23:43:26.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.359+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaf800 con 0x560c2eee9800 2024-09-13T23:43:26.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.363+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.363+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94610326180224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efafc00 con 0x560c2eee9800 2024-09-13T23:43:26.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.363+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaec00 con 0x560c2eee9800 2024-09-13T23:43:26.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.363+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94610326179072] v33'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.363+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94610326180224] v33'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.363+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.363+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94610326181376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaf000 con 0x560c2eee9800 2024-09-13T23:43:26.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.363+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efae400 con 0x560c2eee9c00 2024-09-13T23:43:26.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94610326181376] v33'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'27 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94610326182528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaf800 con 0x560c2eee9c00 2024-09-13T23:43:26.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f2b0bf40 20 add_watcher() i=0 2024-09-13T23:43:26.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f2b0bf40 20 add_watcher() i=1 2024-09-13T23:43:26.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f2b0bf40 20 add_watcher() i=2 2024-09-13T23:43:26.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f2b0bf40 20 add_watcher() i=3 2024-09-13T23:43:26.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f2b0bf40 20 add_watcher() i=4 2024-09-13T23:43:26.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f2b0bf40 20 add_watcher() i=5 2024-09-13T23:43:26.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f2b0bf40 20 add_watcher() i=6 2024-09-13T23:43:26.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94610326182528] v33'28 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f2b0bf40 20 add_watcher() i=7 2024-09-13T23:43:26.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f2b0bf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:26.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efafc00 con 0x560c2eee9000 2024-09-13T23:43:26.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.367+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94610327846912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efaec00 con 0x560c2eee9000 2024-09-13T23:43:26.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.371+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94610327846912] v33'52 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.371+0000 7f60d8292700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:26.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.371+0000 7f60f2b0bf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:26.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.371+0000 7f60f2b0bf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:26.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.371+0000 7f60f2b0bf40 1 Processor -- start 2024-09-13T23:43:26.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.371+0000 7f60f2b0bf40 1 -- start start 2024-09-13T23:43:26.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.371+0000 7f60f2b0bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 0x560c2ee2f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.371+0000 7f60f2b0bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2ebd43c0 con 0x560c2efaf400 2024-09-13T23:43:26.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.371+0000 7f60f0ac3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 0x560c2ee2f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.371+0000 7f60f0ac3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 0x560c2ee2f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43492/0 (socket says 172.21.15.167:43492) 2024-09-13T23:43:26.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.371+0000 7f60f0ac3700 1 -- 172.21.15.167:0/1569074264 learned_addr learned my addr 172.21.15.167:0/1569074264 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.371+0000 7f60f0ac3700 1 -- 172.21.15.167:0/1569074264 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c2eb58b60 con 0x560c2efaf400 2024-09-13T23:43:26.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f0ac3700 1 --2- 172.21.15.167:0/1569074264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 0x560c2ee2f180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x560c2efc0660 tx=0x560c2ef9a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c404cb591d7ad3e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60d7a91700 1 -- 172.21.15.167:0/1569074264 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2efcc000 con 0x560c2efaf400 2024-09-13T23:43:26.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60d7a91700 1 -- 172.21.15.167:0/1569074264 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c2eef6600 con 0x560c2efaf400 2024-09-13T23:43:26.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60d7a91700 1 -- 172.21.15.167:0/1569074264 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2efcc160 con 0x560c2efaf400 2024-09-13T23:43:26.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/1569074264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 msgr2=0x560c2ee2f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/1569074264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 0x560c2ee2f180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x560c2efc0660 tx=0x560c2ef9a060 comp rx=0 tx=0).stop 2024-09-13T23:43:26.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/1569074264 shutdown_connections 2024-09-13T23:43:26.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/1569074264 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 0x560c2ee2f180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/1569074264 >> 172.21.15.167:0/1569074264 conn(0x560c2efae800 msgr2=0x560c2ec48000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/1569074264 shutdown_connections 2024-09-13T23:43:26.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/1569074264 wait complete. 2024-09-13T23:43:26.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f2b0bf40 1 Processor -- start 2024-09-13T23:43:26.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f2b0bf40 1 -- start start 2024-09-13T23:43:26.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f2b0bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 0x560c2ee2f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f2b0bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2ebd4500 con 0x560c2efaf400 2024-09-13T23:43:26.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f0ac3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 0x560c2ee2f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f0ac3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 0x560c2ee2f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43498/0 (socket says 172.21.15.167:43498) 2024-09-13T23:43:26.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f0ac3700 1 -- 172.21.15.167:0/2714468071 learned_addr learned my addr 172.21.15.167:0/2714468071 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f0ac3700 1 -- 172.21.15.167:0/2714468071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c2eb58d00 con 0x560c2efaf400 2024-09-13T23:43:26.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60f0ac3700 1 --2- 172.21.15.167:0/2714468071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 0x560c2ee2f180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x560c2efc1320 tx=0x560c2ef9a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.375+0000 7f60d628e700 1 -- 172.21.15.167:0/2714468071 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2efcc2c0 con 0x560c2efaf400 2024-09-13T23:43:26.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.379+0000 7f60d628e700 1 -- 172.21.15.167:0/2714468071 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560c2eef6300 con 0x560c2efaf400 2024-09-13T23:43:26.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.379+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2714468071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560c2eb58340 con 0x560c2efaf400 2024-09-13T23:43:26.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.379+0000 7f60d628e700 1 -- 172.21.15.167:0/2714468071 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560c2efcc000 con 0x560c2efaf400 2024-09-13T23:43:26.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.379+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2714468071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c2eb58b60 con 0x560c2efaf400 2024-09-13T23:43:26.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.379+0000 7f60d628e700 1 -- 172.21.15.167:0/2714468071 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x560c2ec41b80 con 0x560c2efaf400 2024-09-13T23:43:26.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.379+0000 7f60d628e700 1 --2- 172.21.15.167:0/2714468071 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2f048000 0x560c2f04e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.379+0000 7f60d628e700 1 -- 172.21.15.167:0/2714468071 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x560c2ebad500 con 0x560c2efaf400 2024-09-13T23:43:26.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.379+0000 7f60f2b0bf40 5 note: GC not initialized 2024-09-13T23:43:26.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.379+0000 7f60f1ac5700 1 --2- 172.21.15.167:0/2714468071 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2f048000 0x560c2f04e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.379+0000 7f60f1ac5700 1 --2- 172.21.15.167:0/2714468071 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2f048000 0x560c2f04e000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x560c2ef57590 tx=0x560c2eeab500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.379+0000 7f60d3a89700 20 reqs_thread_entry: start 2024-09-13T23:43:26.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.379+0000 7f60f2b0bf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:26.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:26.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60d1a85700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:56 2024 2024-09-13T23:43:26.392 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:26.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60d1263700 20 reqs_thread_entry: start 2024-09-13T23:43:26.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-13T23:43:26.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2efafc00 con 0x560c2eee9c00 2024-09-13T23:43:26.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-09-13T23:43:26.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:43:26.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-09-13T23:43:26.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2f048400 con 0x560c2eee9000 2024-09-13T23:43:26.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(63 foo@foo.com [stat out=16b,read 0~7 out=7b] v0'0 uv1 ondisk = 0) v8 ==== 197+0+23 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 20 rados_obj.operate() r=0 bl.length=7 2024-09-13T23:43:26.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x5 2024-09-13T23:43:26.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-09-13T23:43:26.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:26.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2f048800 con 0x560c2eee9000 2024-09-13T23:43:26.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(64 foo [call out=48b,read 0~357 out=357b] v0'0 uv1 ondisk = 0) v8 ==== 189+0+405 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:26.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x11 2024-09-13T23:43:26.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:26.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.383+0000 7f60f2b0bf40 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+foo 2024-09-13T23:43:26.395 INFO:teuthology.orchestra.run.smithi167.stderr:could not create user: unable to parse parameters, email: foo@foo.com is the email address of an existing user 2024-09-13T23:43:26.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.387+0000 7f60d8292700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:26.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.387+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94610327846912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2f048c00 con 0x560c2eee9000 2024-09-13T23:43:26.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.387+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94610327846912] v33'53 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.387+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94610326174464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2f049000 con 0x560c2eee9400 2024-09-13T23:43:26.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.391+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94610326174464] v33'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.391+0000 7f60f2b0bf40 20 remove_watcher() i=0 2024-09-13T23:43:26.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.391+0000 7f60f2b0bf40 2 removed watcher, disabling cache 2024-09-13T23:43:26.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.391+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94610326175616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2f049400 con 0x560c2eee9000 2024-09-13T23:43:26.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.391+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94610326175616] v33'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9000 2024-09-13T23:43:26.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.391+0000 7f60f2b0bf40 20 remove_watcher() i=1 2024-09-13T23:43:26.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.391+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94610326176768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2f049800 con 0x560c2eee9400 2024-09-13T23:43:26.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.395+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94610326176768] v33'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.395+0000 7f60f2b0bf40 20 remove_watcher() i=2 2024-09-13T23:43:26.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.395+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94610326177920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2f049c00 con 0x560c2eee9c00 2024-09-13T23:43:26.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.395+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94610326177920] v33'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.395+0000 7f60f2b0bf40 20 remove_watcher() i=3 2024-09-13T23:43:26.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.395+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94610326179072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2f048400 con 0x560c2eee9400 2024-09-13T23:43:26.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.399+0000 7f60f12c4700 1 -- 172.21.15.167:0/3380482707 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94610326179072] v33'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3200 con 0x560c2eee9400 2024-09-13T23:43:26.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.399+0000 7f60f2b0bf40 20 remove_watcher() i=4 2024-09-13T23:43:26.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.399+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94610326180224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2f048800 con 0x560c2eee9800 2024-09-13T23:43:26.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.403+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94610326180224] v33'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.403+0000 7f60f2b0bf40 20 remove_watcher() i=5 2024-09-13T23:43:26.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.403+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94610326181376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2f048c00 con 0x560c2eee9800 2024-09-13T23:43:26.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.403+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94610326181376] v33'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa3440 con 0x560c2eee9800 2024-09-13T23:43:26.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.403+0000 7f60f2b0bf40 20 remove_watcher() i=6 2024-09-13T23:43:26.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.403+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94610326182528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560c2f049400 con 0x560c2eee9c00 2024-09-13T23:43:26.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f1ac5700 1 -- 172.21.15.167:0/3380482707 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94610326182528] v33'29 uv27 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560c2efa2b40 con 0x560c2eee9c00 2024-09-13T23:43:26.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 20 remove_watcher() i=7 2024-09-13T23:43:26.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560c2eee9400 msgr2=0x560c2ee2db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560c2eee9400 0x560c2ee2db80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560c2eee9000 msgr2=0x560c2ee2d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560c2eee9000 0x560c2ee2d600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560c2eee9c00 msgr2=0x560c2ee2c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560c2eee9c00 0x560c2ee2c580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560c2eee9800 msgr2=0x560c2ee2ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560c2eee9800 0x560c2ee2ec00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2eee8c00 msgr2=0x560c2ee2e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2eee8c00 0x560c2ee2e680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x560c2ee27f80 tx=0x560c2ef9a0c0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 msgr2=0x560c2ee2d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f0ac3700 1 -- 172.21.15.167:0/3380482707 reap_dead start 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 0x560c2ee2d080 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x560c2ee6a7b0 tx=0x560c2ec4bce0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 shutdown_connections 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/3380482707 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47c00 0x560c2ee2d080 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 >> 172.21.15.167:0/3380482707 conn(0x560c2eb47800 msgr2=0x560c2ec47800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 shutdown_connections 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/3380482707 wait complete. 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2714468071 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2f048000 msgr2=0x560c2f04e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/2714468071 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2f048000 0x560c2f04e000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x560c2ef57590 tx=0x560c2eeab500 comp rx=0 tx=0).stop 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2714468071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 msgr2=0x560c2ee2f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/2714468071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 0x560c2ee2f180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x560c2efc1320 tx=0x560c2ef9a180 comp rx=0 tx=0).stop 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2714468071 shutdown_connections 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/2714468071 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2f048000 0x560c2f04e000 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/2714468071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2efaf400 0x560c2ee2f180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2714468071 >> 172.21.15.167:0/2714468071 conn(0x560c2efae800 msgr2=0x560c2ec48000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2714468071 shutdown_connections 2024-09-13T23:43:26.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/2714468071 wait complete. 2024-09-13T23:43:26.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/578733959 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2eee8000 msgr2=0x560c2ee2e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/578733959 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2eee8000 0x560c2ee2e100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x560c2ee6ae40 tx=0x560c2eeaab40 comp rx=0 tx=0).stop 2024-09-13T23:43:26.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/578733959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47400 msgr2=0x560c2ee2cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/578733959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47400 0x560c2ee2cb00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x560c2eca20f0 tx=0x560c2ec4b920 comp rx=0 tx=0).stop 2024-09-13T23:43:26.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/578733959 shutdown_connections 2024-09-13T23:43:26.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/578733959 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560c2eee8000 0x560c2ee2e100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 --2- 172.21.15.167:0/578733959 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560c2eb47400 0x560c2ee2cb00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.407+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/578733959 >> 172.21.15.167:0/578733959 conn(0x560c2eb46c00 msgr2=0x560c2ec47000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.411+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/578733959 shutdown_connections 2024-09-13T23:43:26.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.411+0000 7f60f2b0bf40 1 -- 172.21.15.167:0/578733959 wait complete. 2024-09-13T23:43:26.423 DEBUG:teuthology.orchestra.run:got remote process result: 240 2024-09-13T23:43:26.424 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-13T23:43:26.424 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-13T23:43:26.424 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-13T23:43:26.465 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:26.465 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:26.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.455+0000 7f34776b3f40 1 Processor -- start 2024-09-13T23:43:26.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.455+0000 7f34776b3f40 1 -- start start 2024-09-13T23:43:26.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.455+0000 7f34776b3f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c57563f000 0x55c575924580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.455+0000 7f34776b3f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c575703500 con 0x55c57563f400 2024-09-13T23:43:26.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.455+0000 7f34776b3f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c5756cca00 con 0x55c57563f000 2024-09-13T23:43:26.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f3475e6c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c57563f000 0x55c575924580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f347566b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55c57563f400 0x55c57573e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45216/0 (socket says 172.21.15.167:45216) 2024-09-13T23:43:26.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f347566b700 1 -- 172.21.15.167:0/2280741927 learned_addr learned my addr 172.21.15.167:0/2280741927 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:26.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f3475e6c700 1 -- 172.21.15.167:0/2280741927 >> v1:172.21.15.12:6789/0 conn(0x55c57563f400 legacy=0x55c57573e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f3475e6c700 1 -- 172.21.15.167:0/2280741927 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c575651d40 con 0x55c57563f000 2024-09-13T23:43:26.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f3475e6c700 1 --2- 172.21.15.167:0/2280741927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55c57591fa70 tx=0x55c575743920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1da677ccc4f8f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f3474e6a700 1 -- 172.21.15.167:0/2280741927 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5756e34a0 con 0x55c57563f000 2024-09-13T23:43:26.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f3474e6a700 1 -- 172.21.15.167:0/2280741927 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c575703800 con 0x55c57563f000 2024-09-13T23:43:26.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f3474e6a700 1 -- 172.21.15.167:0/2280741927 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5756e3600 con 0x55c57563f000 2024-09-13T23:43:26.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f34776b3f40 1 -- 172.21.15.167:0/2280741927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 msgr2=0x55c575924580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2280741927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x55c57591fa70 tx=0x55c575743920 comp rx=0 tx=0).stop 2024-09-13T23:43:26.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f34776b3f40 1 -- 172.21.15.167:0/2280741927 shutdown_connections 2024-09-13T23:43:26.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2280741927 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f34776b3f40 1 -- 172.21.15.167:0/2280741927 >> 172.21.15.167:0/2280741927 conn(0x55c57563ec00 msgr2=0x55c57573e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f34776b3f40 1 -- 172.21.15.167:0/2280741927 shutdown_connections 2024-09-13T23:43:26.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f34776b3f40 1 -- 172.21.15.167:0/2280741927 wait complete. 2024-09-13T23:43:26.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f34776b3f40 1 Processor -- start 2024-09-13T23:43:26.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f34776b3f40 1 -- start start 2024-09-13T23:43:26.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f34776b3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f34776b3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5756cc500 con 0x55c57563f000 2024-09-13T23:43:26.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f3475e6c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f3475e6c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50504/0 (socket says 172.21.15.167:50504) 2024-09-13T23:43:26.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.459+0000 7f3475e6c700 1 -- 172.21.15.167:0/2079895178 learned_addr learned my addr 172.21.15.167:0/2079895178 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f3475e6c700 1 -- 172.21.15.167:0/2079895178 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c575651ba0 con 0x55c57563f000 2024-09-13T23:43:26.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f3475e6c700 1 --2- 172.21.15.167:0/2079895178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55c57591eb10 tx=0x55c575743800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a10c9b060105d61 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f3474669700 1 -- 172.21.15.167:0/2079895178 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5756e3760 con 0x55c57563f000 2024-09-13T23:43:26.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f3474669700 1 -- 172.21.15.167:0/2079895178 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c575703b00 con 0x55c57563f000 2024-09-13T23:43:26.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f3474669700 1 -- 172.21.15.167:0/2079895178 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5756e34a0 con 0x55c57563f000 2024-09-13T23:43:26.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f34776b3f40 1 -- 172.21.15.167:0/2079895178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 msgr2=0x55c575924580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2079895178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55c57591eb10 tx=0x55c575743800 comp rx=0 tx=0).stop 2024-09-13T23:43:26.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f34776b3f40 1 -- 172.21.15.167:0/2079895178 shutdown_connections 2024-09-13T23:43:26.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2079895178 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f34776b3f40 1 -- 172.21.15.167:0/2079895178 >> 172.21.15.167:0/2079895178 conn(0x55c57563ec00 msgr2=0x55c57573f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f34776b3f40 1 -- 172.21.15.167:0/2079895178 shutdown_connections 2024-09-13T23:43:26.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f34776b3f40 1 -- 172.21.15.167:0/2079895178 wait complete. 2024-09-13T23:43:26.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f34776b3f40 1 Processor -- start 2024-09-13T23:43:26.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f34776b3f40 1 -- start start 2024-09-13T23:43:26.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f34776b3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f34776b3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5756cc3c0 con 0x55c57563f000 2024-09-13T23:43:26.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f3475e6c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f3475e6c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50518/0 (socket says 172.21.15.167:50518) 2024-09-13T23:43:26.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f3475e6c700 1 -- 172.21.15.167:0/211405591 learned_addr learned my addr 172.21.15.167:0/211405591 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f3475e6c700 1 -- 172.21.15.167:0/211405591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c575651a00 con 0x55c57563f000 2024-09-13T23:43:26.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.463+0000 7f3475e6c700 1 --2- 172.21.15.167:0/211405591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55c57591fe90 tx=0x55c575743920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f3472e66700 1 -- 172.21.15.167:0/211405591 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5756e3600 con 0x55c57563f000 2024-09-13T23:43:26.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f3472e66700 1 -- 172.21.15.167:0/211405591 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c575703c80 con 0x55c57563f000 2024-09-13T23:43:26.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f34776b3f40 1 -- 172.21.15.167:0/211405591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c575651860 con 0x55c57563f000 2024-09-13T23:43:26.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f3472e66700 1 -- 172.21.15.167:0/211405591 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5756e38c0 con 0x55c57563f000 2024-09-13T23:43:26.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f34776b3f40 1 -- 172.21.15.167:0/211405591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5756516c0 con 0x55c57563f000 2024-09-13T23:43:26.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f34776b3f40 1 Processor -- start 2024-09-13T23:43:26.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f34776b3f40 1 -- start start 2024-09-13T23:43:26.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f34776b3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 0x55c575924b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f34776b3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5756cc000 con 0x55c57563f800 2024-09-13T23:43:26.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f3472e66700 1 -- 172.21.15.167:0/211405591 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c575738b00 con 0x55c57563f000 2024-09-13T23:43:26.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f3472e66700 1 --2- 172.21.15.167:0/211405591 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c5759de000 0x55c575925080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f3472e66700 1 -- 172.21.15.167:0/211405591 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c5756a5180 con 0x55c57563f000 2024-09-13T23:43:26.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f347666d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 0x55c575924b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f347666d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 0x55c575924b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50532/0 (socket says 172.21.15.167:50532) 2024-09-13T23:43:26.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f347666d700 1 -- 172.21.15.167:0/2031049440 learned_addr learned my addr 172.21.15.167:0/2031049440 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f3475e6c700 1 --2- 172.21.15.167:0/211405591 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c5759de000 0x55c575925080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f347666d700 1 -- 172.21.15.167:0/2031049440 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c575650000 con 0x55c57563f800 2024-09-13T23:43:26.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f3475e6c700 1 --2- 172.21.15.167:0/211405591 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c5759de000 0x55c575925080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55c5759674d0 tx=0x55c575998b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f347666d700 1 --2- 172.21.15.167:0/2031049440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 0x55c575924b00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55c575966b40 tx=0x55c575998e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db48aa01921ab06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f3470661700 1 -- 172.21.15.167:0/2031049440 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5756e3600 con 0x55c57563f800 2024-09-13T23:43:26.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.467+0000 7f3470661700 1 -- 172.21.15.167:0/2031049440 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c5759ee000 con 0x55c57563f800 2024-09-13T23:43:26.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f3470661700 1 -- 172.21.15.167:0/2031049440 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5756e38c0 con 0x55c57563f800 2024-09-13T23:43:26.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 -- 172.21.15.167:0/2031049440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 msgr2=0x55c575924b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2031049440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 0x55c575924b00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55c575966b40 tx=0x55c575998e40 comp rx=0 tx=0).stop 2024-09-13T23:43:26.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 -- 172.21.15.167:0/2031049440 shutdown_connections 2024-09-13T23:43:26.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2031049440 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 0x55c575924b00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 -- 172.21.15.167:0/2031049440 >> 172.21.15.167:0/2031049440 conn(0x55c57563f400 msgr2=0x55c57573f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 -- 172.21.15.167:0/2031049440 shutdown_connections 2024-09-13T23:43:26.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 -- 172.21.15.167:0/2031049440 wait complete. 2024-09-13T23:43:26.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 Processor -- start 2024-09-13T23:43:26.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 -- start start 2024-09-13T23:43:26.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 0x55c575924b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c575664640 con 0x55c57563f800 2024-09-13T23:43:26.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f347666d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 0x55c575924b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f347666d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 0x55c575924b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50546/0 (socket says 172.21.15.167:50546) 2024-09-13T23:43:26.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 learned_addr learned my addr 172.21.15.167:0/2086516330 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c575650ea0 con 0x55c57563f800 2024-09-13T23:43:26.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f347666d700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 0x55c575924b00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55c575966810 tx=0x55c575998a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f346ee5e700 1 -- 172.21.15.167:0/2086516330 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5756e3760 con 0x55c57563f800 2024-09-13T23:43:26.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c575650820 con 0x55c57563f800 2024-09-13T23:43:26.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f346ee5e700 1 -- 172.21.15.167:0/2086516330 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c5759ee180 con 0x55c57563f800 2024-09-13T23:43:26.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f346ee5e700 1 -- 172.21.15.167:0/2086516330 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c5756e3600 con 0x55c57563f800 2024-09-13T23:43:26.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.471+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5756509c0 con 0x55c57563f800 2024-09-13T23:43:26.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f34776b3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f346ee5e700 1 -- 172.21.15.167:0/2086516330 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c575739080 con 0x55c57563f800 2024-09-13T23:43:26.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f346ee5e700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c5759dec00 0x55c575925600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f346ee5e700 1 -- 172.21.15.167:0/2086516330 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c5756a5340 con 0x55c57563f800 2024-09-13T23:43:26.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f347566b700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c5759dec00 0x55c575925600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c5759df000 0x55c575925b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df400 con 0x55c5759df000 2024-09-13T23:43:26.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f347666d700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c5759df000 0x55c575925b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f347566b700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c5759dec00 0x55c575925600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55c5759c1b00 tx=0x55c575a980c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f347666d700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c5759df000 0x55c575925b80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f34776b3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f34776b3f40 20 realm 2024-09-13T23:43:26.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f34776b3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df000 2024-09-13T23:43:26.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f34776b3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f34776b3f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:26.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f34776b3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.475+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759dfc00 con 0x55c5759df000 2024-09-13T23:43:26.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f34776b3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f34776b3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c5759df400 0x55c575926100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df400 2024-09-13T23:43:26.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f3475e6c700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c5759df400 0x55c575926100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f3475e6c700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c5759df400 0x55c575926100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f34776b3f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:26.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f34776b3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759dfc00 con 0x55c5759df400 2024-09-13T23:43:26.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f34776b3f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:26.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f34776b3f40 20 searching for the correct realm 2024-09-13T23:43:26.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c575980000 0x55c575926680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575980400 con 0x55c575980000 2024-09-13T23:43:26.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f347566b700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c575980000 0x55c575926680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.479+0000 7f347566b700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c575980000 0x55c575926680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575980400 con 0x55c5759df400 2024-09-13T23:43:26.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f3475e6c700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c575980400 0x55c575926c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980400 2024-09-13T23:43:26.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f347666d700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c575980400 0x55c575926c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f347666d700 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c575980400 0x55c575926c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df000 2024-09-13T23:43:26.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df000 2024-09-13T23:43:26.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.483+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980000 2024-09-13T23:43:26.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df000 2024-09-13T23:43:26.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df400 2024-09-13T23:43:26.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df400 2024-09-13T23:43:26.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df000 2024-09-13T23:43:26.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980400 2024-09-13T23:43:26.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df000 2024-09-13T23:43:26.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.487+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980400 2024-09-13T23:43:26.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.491+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.491+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df400 2024-09-13T23:43:26.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.491+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.491+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980000 2024-09-13T23:43:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.491+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.491+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df000 2024-09-13T23:43:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.491+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.491+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df400 2024-09-13T23:43:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.491+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.491+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980000 2024-09-13T23:43:26.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.491+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.491+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df000 2024-09-13T23:43:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.495+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.495+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df000 2024-09-13T23:43:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.495+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.495+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df000 2024-09-13T23:43:26.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.495+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.495+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980000 2024-09-13T23:43:26.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.495+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.495+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980400 2024-09-13T23:43:26.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.495+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.495+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df000 2024-09-13T23:43:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.495+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.495+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980400 2024-09-13T23:43:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.499+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.499+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980000 2024-09-13T23:43:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.499+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.499+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980400 2024-09-13T23:43:26.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.499+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.499+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df400 2024-09-13T23:43:26.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.499+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.499+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df400 2024-09-13T23:43:26.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.499+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.499+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980000 2024-09-13T23:43:26.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.499+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.499+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df400 2024-09-13T23:43:26.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df400 2024-09-13T23:43:26.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:26.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:26.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:26.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:26.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575980800 con 0x55c5759df000 2024-09-13T23:43:26.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575980c00 con 0x55c5759df400 2024-09-13T23:43:26.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:26.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575981000 con 0x55c5759df400 2024-09-13T23:43:26.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:26.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 zone default found 2024-09-13T23:43:26.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 4 Realm: () 2024-09-13T23:43:26.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:26.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:26.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:26.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 zonegroup default 2024-09-13T23:43:26.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.503+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575981400 con 0x55c575980400 2024-09-13T23:43:26.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f34776b3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f34776b3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575981800 con 0x55c575980000 2024-09-13T23:43:26.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f34776b3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f34776b3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575981c00 con 0x55c575980000 2024-09-13T23:43:26.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f34776b3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f34776b3f40 20 started sync module instance, tier type = 2024-09-13T23:43:26.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f34776b3f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:26.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.507+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575981800 con 0x55c5759df400 2024-09-13T23:43:26.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.511+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.511+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94306569403136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575981c00 con 0x55c5759df400 2024-09-13T23:43:26.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.511+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575980c00 con 0x55c5759df000 2024-09-13T23:43:26.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.511+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94306569403136] v33'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.511+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.511+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94306569404288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575981000 con 0x55c5759df000 2024-09-13T23:43:26.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.511+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575981800 con 0x55c5759df400 2024-09-13T23:43:26.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.515+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94306569404288] v33'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.515+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.515+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94306569405440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759dfc00 con 0x55c5759df400 2024-09-13T23:43:26.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.515+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c575980400 2024-09-13T23:43:26.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.519+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.519+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94306569406592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575980800 con 0x55c575980400 2024-09-13T23:43:26.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.519+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575981400 con 0x55c5759df400 2024-09-13T23:43:26.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.519+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94306569405440] v33'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.519+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94306569406592] v33'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.519+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.519+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94306569407744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575980c00 con 0x55c5759df400 2024-09-13T23:43:26.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.519+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575981000 con 0x55c575980000 2024-09-13T23:43:26.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.523+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.523+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94306569407744] v33'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.523+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94306569408896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575982000 con 0x55c575980000 2024-09-13T23:43:26.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.523+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575982400 con 0x55c575980000 2024-09-13T23:43:26.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94306569410048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575982800 con 0x55c575980000 2024-09-13T23:43:26.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575982c00 con 0x55c575980400 2024-09-13T23:43:26.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94306569408896] v33'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'30 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94306569411200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575983000 con 0x55c575980400 2024-09-13T23:43:26.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f34776b3f40 20 add_watcher() i=0 2024-09-13T23:43:26.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f34776b3f40 20 add_watcher() i=1 2024-09-13T23:43:26.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f34776b3f40 20 add_watcher() i=2 2024-09-13T23:43:26.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f34776b3f40 20 add_watcher() i=3 2024-09-13T23:43:26.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f34776b3f40 20 add_watcher() i=4 2024-09-13T23:43:26.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f34776b3f40 20 add_watcher() i=5 2024-09-13T23:43:26.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94306569410048] v33'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f34776b3f40 20 add_watcher() i=6 2024-09-13T23:43:26.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94306569411200] v33'31 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f34776b3f40 20 add_watcher() i=7 2024-09-13T23:43:26.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.527+0000 7f34776b3f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:26.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.531+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575983400 con 0x55c5759df000 2024-09-13T23:43:26.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.531+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.531+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94306571075584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575983800 con 0x55c5759df000 2024-09-13T23:43:26.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.531+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94306571075584] v33'54 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.531+0000 7f345ce3a700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:26.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.531+0000 7f34776b3f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:26.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.531+0000 7f34776b3f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:26.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.531+0000 7f34776b3f40 1 Processor -- start 2024-09-13T23:43:26.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.531+0000 7f34776b3f40 1 -- start start 2024-09-13T23:43:26.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.531+0000 7f34776b3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 0x55c575927180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.531+0000 7f34776b3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c575664780 con 0x55c575982c00 2024-09-13T23:43:26.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.531+0000 7f347566b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 0x55c575927180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f347566b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 0x55c575927180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50554/0 (socket says 172.21.15.167:50554) 2024-09-13T23:43:26.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f347566b700 1 -- 172.21.15.167:0/989492697 learned_addr learned my addr 172.21.15.167:0/989492697 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f347566b700 1 -- 172.21.15.167:0/989492697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c575650b60 con 0x55c575982c00 2024-09-13T23:43:26.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f347566b700 1 --2- 172.21.15.167:0/989492697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 0x55c575927180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55c575abe030 tx=0x55c575a98060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad351eaec010e8cc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f345c639700 1 -- 172.21.15.167:0/989492697 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c575ac6000 con 0x55c575982c00 2024-09-13T23:43:26.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f345c639700 1 -- 172.21.15.167:0/989492697 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c5759ee600 con 0x55c575982c00 2024-09-13T23:43:26.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f345c639700 1 -- 172.21.15.167:0/989492697 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c575ac6160 con 0x55c575982c00 2024-09-13T23:43:26.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f34776b3f40 1 -- 172.21.15.167:0/989492697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 msgr2=0x55c575927180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f34776b3f40 1 --2- 172.21.15.167:0/989492697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 0x55c575927180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55c575abe030 tx=0x55c575a98060 comp rx=0 tx=0).stop 2024-09-13T23:43:26.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f34776b3f40 1 -- 172.21.15.167:0/989492697 shutdown_connections 2024-09-13T23:43:26.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f34776b3f40 1 --2- 172.21.15.167:0/989492697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 0x55c575927180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f34776b3f40 1 -- 172.21.15.167:0/989492697 >> 172.21.15.167:0/989492697 conn(0x55c575983c00 msgr2=0x55c575740000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f34776b3f40 1 -- 172.21.15.167:0/989492697 shutdown_connections 2024-09-13T23:43:26.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f34776b3f40 1 -- 172.21.15.167:0/989492697 wait complete. 2024-09-13T23:43:26.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f34776b3f40 1 Processor -- start 2024-09-13T23:43:26.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f34776b3f40 1 -- start start 2024-09-13T23:43:26.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f34776b3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 0x55c575927180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f34776b3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5756cc000 con 0x55c575982c00 2024-09-13T23:43:26.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f347566b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 0x55c575927180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f347566b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 0x55c575927180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50562/0 (socket says 172.21.15.167:50562) 2024-09-13T23:43:26.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.535+0000 7f347566b700 1 -- 172.21.15.167:0/3420265973 learned_addr learned my addr 172.21.15.167:0/3420265973 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f347566b700 1 -- 172.21.15.167:0/3420265973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c575650d00 con 0x55c575982c00 2024-09-13T23:43:26.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f347566b700 1 --2- 172.21.15.167:0/3420265973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 0x55c575927180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55c575abe270 tx=0x55c575a98180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f345ae36700 1 -- 172.21.15.167:0/3420265973 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c575ac62c0 con 0x55c575982c00 2024-09-13T23:43:26.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f345ae36700 1 -- 172.21.15.167:0/3420265973 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c5759ee300 con 0x55c575982c00 2024-09-13T23:43:26.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f34776b3f40 1 -- 172.21.15.167:0/3420265973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c575650340 con 0x55c575982c00 2024-09-13T23:43:26.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f345ae36700 1 -- 172.21.15.167:0/3420265973 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c575ac6000 con 0x55c575982c00 2024-09-13T23:43:26.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f34776b3f40 1 -- 172.21.15.167:0/3420265973 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c575650b60 con 0x55c575982c00 2024-09-13T23:43:26.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f345ae36700 1 -- 172.21.15.167:0/3420265973 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c575739b80 con 0x55c575982c00 2024-09-13T23:43:26.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f345ae36700 1 --2- 172.21.15.167:0/3420265973 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c575b3e000 0x55c575b44000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f345ae36700 1 -- 172.21.15.167:0/3420265973 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c5756a5500 con 0x55c575982c00 2024-09-13T23:43:26.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f347666d700 1 --2- 172.21.15.167:0/3420265973 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c575b3e000 0x55c575b44000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f34776b3f40 5 note: GC not initialized 2024-09-13T23:43:26.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.539+0000 7f347666d700 1 --2- 172.21.15.167:0/3420265973 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c575b3e000 0x55c575b44000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55c575a51590 tx=0x55c575999500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.543+0000 7f3458631700 20 reqs_thread_entry: start 2024-09-13T23:43:26.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.543+0000 7f34776b3f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:26.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.543+0000 7f34776b3f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:26.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.543+0000 7f345662d700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:56 2024 2024-09-13T23:43:26.552 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:26.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.543+0000 7f3455e0b700 20 reqs_thread_entry: start 2024-09-13T23:43:26.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.543+0000 7f34776b3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.543+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575982000 con 0x55c5759df000 2024-09-13T23:43:26.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.543+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.543+0000 7f34776b3f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:26.553 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:26.553 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:26.553 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:26.553 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:26.554 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:26.555 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:26.556 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:26.556 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:26.556 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:26.556 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:26.556 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:26.556 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:26.556 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:26.556 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:26.556 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:26.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.543+0000 7f345ce3a700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:26.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.543+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94306571075584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575982400 con 0x55c5759df000 2024-09-13T23:43:26.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.547+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94306571075584] v33'55 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.547+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94306569403136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575980800 con 0x55c5759df400 2024-09-13T23:43:26.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.547+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94306569403136] v33'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.547+0000 7f34776b3f40 20 remove_watcher() i=0 2024-09-13T23:43:26.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.547+0000 7f34776b3f40 2 removed watcher, disabling cache 2024-09-13T23:43:26.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.551+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94306569404288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575981000 con 0x55c5759df000 2024-09-13T23:43:26.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.551+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94306569404288] v33'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c5759df000 2024-09-13T23:43:26.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.551+0000 7f34776b3f40 20 remove_watcher() i=1 2024-09-13T23:43:26.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.551+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94306569405440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c5759df800 con 0x55c5759df400 2024-09-13T23:43:26.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.555+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94306569405440] v33'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.555+0000 7f34776b3f40 20 remove_watcher() i=2 2024-09-13T23:43:26.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.555+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94306569406592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575b3e400 con 0x55c575980400 2024-09-13T23:43:26.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.555+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94306569406592] v33'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.555+0000 7f34776b3f40 20 remove_watcher() i=3 2024-09-13T23:43:26.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.555+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94306569407744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575b3e800 con 0x55c5759df400 2024-09-13T23:43:26.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.559+0000 7f3475e6c700 1 -- 172.21.15.167:0/2086516330 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94306569407744] v33'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87200 con 0x55c5759df400 2024-09-13T23:43:26.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.559+0000 7f34776b3f40 20 remove_watcher() i=4 2024-09-13T23:43:26.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.559+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94306569408896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575b3ec00 con 0x55c575980000 2024-09-13T23:43:26.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.563+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94306569408896] v33'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.563+0000 7f34776b3f40 20 remove_watcher() i=5 2024-09-13T23:43:26.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.563+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94306569410048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575b3f000 con 0x55c575980000 2024-09-13T23:43:26.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.563+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94306569410048] v33'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a87440 con 0x55c575980000 2024-09-13T23:43:26.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.563+0000 7f34776b3f40 20 remove_watcher() i=6 2024-09-13T23:43:26.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.563+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94306569411200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c575b3f400 con 0x55c575980400 2024-09-13T23:43:26.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f347666d700 1 -- 172.21.15.167:0/2086516330 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94306569411200] v33'32 uv30 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c575a86b40 con 0x55c575980400 2024-09-13T23:43:26.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 20 remove_watcher() i=7 2024-09-13T23:43:26.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c5759df400 msgr2=0x55c575926100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c5759df400 0x55c575926100 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c5759df000 msgr2=0x55c575925b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c5759df000 0x55c575925b80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c575980400 msgr2=0x55c575926c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c575980400 0x55c575926c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c575980000 msgr2=0x55c575926680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c575980000 0x55c575926680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c5759dec00 msgr2=0x55c575925600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c5759dec00 0x55c575925600 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55c5759c1b00 tx=0x55c575a980c0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 msgr2=0x55c575924b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 0x55c575924b00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55c575966810 tx=0x55c575998a80 comp rx=0 tx=0).stop 2024-09-13T23:43:26.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f347566b700 1 -- 172.21.15.167:0/2086516330 reap_dead start 2024-09-13T23:43:26.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 shutdown_connections 2024-09-13T23:43:26.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 --2- 172.21.15.167:0/2086516330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f800 0x55c575924b00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 >> 172.21.15.167:0/2086516330 conn(0x55c57563f400 msgr2=0x55c57573f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 shutdown_connections 2024-09-13T23:43:26.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/2086516330 wait complete. 2024-09-13T23:43:26.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/3420265973 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c575b3e000 msgr2=0x55c575b44000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 --2- 172.21.15.167:0/3420265973 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c575b3e000 0x55c575b44000 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55c575a51590 tx=0x55c575999500 comp rx=0 tx=0).stop 2024-09-13T23:43:26.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/3420265973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 msgr2=0x55c575927180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 --2- 172.21.15.167:0/3420265973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 0x55c575927180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55c575abe270 tx=0x55c575a98180 comp rx=0 tx=0).stop 2024-09-13T23:43:26.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/3420265973 shutdown_connections 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 --2- 172.21.15.167:0/3420265973 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c575b3e000 0x55c575b44000 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 --2- 172.21.15.167:0/3420265973 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c575982c00 0x55c575927180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/3420265973 >> 172.21.15.167:0/3420265973 conn(0x55c575983c00 msgr2=0x55c575740000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/3420265973 shutdown_connections 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.567+0000 7f34776b3f40 1 -- 172.21.15.167:0/3420265973 wait complete. 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.571+0000 7f34776b3f40 1 -- 172.21.15.167:0/211405591 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c5759de000 msgr2=0x55c575925080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.571+0000 7f34776b3f40 1 --2- 172.21.15.167:0/211405591 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c5759de000 0x55c575925080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x55c5759674d0 tx=0x55c575998b40 comp rx=0 tx=0).stop 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.571+0000 7f34776b3f40 1 -- 172.21.15.167:0/211405591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 msgr2=0x55c575924580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.571+0000 7f34776b3f40 1 --2- 172.21.15.167:0/211405591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55c57591fe90 tx=0x55c575743920 comp rx=0 tx=0).stop 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.571+0000 7f34776b3f40 1 -- 172.21.15.167:0/211405591 shutdown_connections 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.571+0000 7f34776b3f40 1 --2- 172.21.15.167:0/211405591 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c5759de000 0x55c575925080 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.571+0000 7f34776b3f40 1 --2- 172.21.15.167:0/211405591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c57563f000 0x55c575924580 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.571+0000 7f34776b3f40 1 -- 172.21.15.167:0/211405591 >> 172.21.15.167:0/211405591 conn(0x55c57563ec00 msgr2=0x55c57573f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.571+0000 7f34776b3f40 1 -- 172.21.15.167:0/211405591 shutdown_connections 2024-09-13T23:43:26.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.571+0000 7f34776b3f40 1 -- 172.21.15.167:0/211405591 wait complete. 2024-09-13T23:43:26.584 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:26.584 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-09-13T23:43:26.584 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-09-13T23:43:26.585 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-09-13T23:43:26.666 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:26.666 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:26.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa237cc6f40 1 Processor -- start 2024-09-13T23:43:26.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa237cc6f40 1 -- start start 2024-09-13T23:43:26.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa237cc6f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5653229dd400 0x565322cc2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa237cc6f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x565322aa1500 con 0x5653229dd000 2024-09-13T23:43:26.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa237cc6f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x565322a6aa00 con 0x5653229dd400 2024-09-13T23:43:26.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa235c7e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5653229dd400 0x565322cc2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa23647f700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5653229dd000 0x565322adc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45222/0 (socket says 172.21.15.167:45222) 2024-09-13T23:43:26.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa23647f700 1 -- 172.21.15.167:0/197076804 learned_addr learned my addr 172.21.15.167:0/197076804 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:26.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa235c7e700 1 -- 172.21.15.167:0/197076804 >> v1:172.21.15.12:6789/0 conn(0x5653229dd000 legacy=0x565322adc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa235c7e700 1 -- 172.21.15.167:0/197076804 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653229efd40 con 0x5653229dd400 2024-09-13T23:43:26.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa235c7e700 1 --2- 172.21.15.167:0/197076804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd400 0x565322cc2580 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x565322cbd920 tx=0x565322ae1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3db84c5f2a3013 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa23547d700 1 -- 172.21.15.167:0/197076804 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322a814a0 con 0x5653229dd400 2024-09-13T23:43:26.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa23547d700 1 -- 172.21.15.167:0/197076804 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565322aa1800 con 0x5653229dd400 2024-09-13T23:43:26.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa23547d700 1 -- 172.21.15.167:0/197076804 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322a81600 con 0x5653229dd400 2024-09-13T23:43:26.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa237cc6f40 1 -- 172.21.15.167:0/197076804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd400 msgr2=0x565322cc2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/197076804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd400 0x565322cc2580 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x565322cbd920 tx=0x565322ae1920 comp rx=0 tx=0).stop 2024-09-13T23:43:26.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa237cc6f40 1 -- 172.21.15.167:0/197076804 shutdown_connections 2024-09-13T23:43:26.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/197076804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd400 0x565322cc2580 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa237cc6f40 1 -- 172.21.15.167:0/197076804 >> 172.21.15.167:0/197076804 conn(0x5653229dcc00 msgr2=0x565322adc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.659+0000 7fa237cc6f40 1 -- 172.21.15.167:0/197076804 shutdown_connections 2024-09-13T23:43:26.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa237cc6f40 1 -- 172.21.15.167:0/197076804 wait complete. 2024-09-13T23:43:26.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa237cc6f40 1 Processor -- start 2024-09-13T23:43:26.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa237cc6f40 1 -- start start 2024-09-13T23:43:26.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa237cc6f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 0x565322cc2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa237cc6f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x565322a6a500 con 0x5653229dd000 2024-09-13T23:43:26.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa23647f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 0x565322cc2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa23647f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 0x565322cc2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50594/0 (socket says 172.21.15.167:50594) 2024-09-13T23:43:26.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa23647f700 1 -- 172.21.15.167:0/966272041 learned_addr learned my addr 172.21.15.167:0/966272041 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa23647f700 1 -- 172.21.15.167:0/966272041 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653229efba0 con 0x5653229dd000 2024-09-13T23:43:26.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa23647f700 1 --2- 172.21.15.167:0/966272041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 0x565322cc2580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x565322b39e60 tx=0x565322ae1b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5bf975c13ef6ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa234c7c700 1 -- 172.21.15.167:0/966272041 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322a814a0 con 0x5653229dd000 2024-09-13T23:43:26.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa234c7c700 1 -- 172.21.15.167:0/966272041 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565322aa1b00 con 0x5653229dd000 2024-09-13T23:43:26.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.663+0000 7fa234c7c700 1 -- 172.21.15.167:0/966272041 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322a81600 con 0x5653229dd000 2024-09-13T23:43:26.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa237cc6f40 1 -- 172.21.15.167:0/966272041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 msgr2=0x565322cc2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/966272041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 0x565322cc2580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x565322b39e60 tx=0x565322ae1b60 comp rx=0 tx=0).stop 2024-09-13T23:43:26.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa237cc6f40 1 -- 172.21.15.167:0/966272041 shutdown_connections 2024-09-13T23:43:26.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/966272041 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 0x565322cc2580 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa237cc6f40 1 -- 172.21.15.167:0/966272041 >> 172.21.15.167:0/966272041 conn(0x5653229dcc00 msgr2=0x565322add000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa237cc6f40 1 -- 172.21.15.167:0/966272041 shutdown_connections 2024-09-13T23:43:26.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa237cc6f40 1 -- 172.21.15.167:0/966272041 wait complete. 2024-09-13T23:43:26.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa237cc6f40 1 Processor -- start 2024-09-13T23:43:26.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa237cc6f40 1 -- start start 2024-09-13T23:43:26.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa237cc6f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 0x565322cc2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa237cc6f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x565322a6a3c0 con 0x5653229dd000 2024-09-13T23:43:26.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa23647f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 0x565322cc2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa23647f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 0x565322cc2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50606/0 (socket says 172.21.15.167:50606) 2024-09-13T23:43:26.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa23647f700 1 -- 172.21.15.167:0/2865227281 learned_addr learned my addr 172.21.15.167:0/2865227281 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa23647f700 1 -- 172.21.15.167:0/2865227281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653229efa00 con 0x5653229dd000 2024-09-13T23:43:26.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.667+0000 7fa23647f700 1 --2- 172.21.15.167:0/2865227281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 0x565322cc2580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x56532384e0c0 tx=0x565322ae1a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa233479700 1 -- 172.21.15.167:0/2865227281 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322a818c0 con 0x5653229dd000 2024-09-13T23:43:26.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa233479700 1 -- 172.21.15.167:0/2865227281 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565322aa1c80 con 0x5653229dd000 2024-09-13T23:43:26.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa233479700 1 -- 172.21.15.167:0/2865227281 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322a814a0 con 0x5653229dd000 2024-09-13T23:43:26.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa237cc6f40 1 -- 172.21.15.167:0/2865227281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5653229ef860 con 0x5653229dd000 2024-09-13T23:43:26.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa237cc6f40 1 -- 172.21.15.167:0/2865227281 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653229ef6c0 con 0x5653229dd000 2024-09-13T23:43:26.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa237cc6f40 1 Processor -- start 2024-09-13T23:43:26.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa237cc6f40 1 -- start start 2024-09-13T23:43:26.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa237cc6f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 0x565322cc2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa237cc6f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x565322a02640 con 0x5653229dd800 2024-09-13T23:43:26.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa236c80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 0x565322cc2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa236c80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 0x565322cc2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50616/0 (socket says 172.21.15.167:50616) 2024-09-13T23:43:26.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa236c80700 1 -- 172.21.15.167:0/3339740892 learned_addr learned my addr 172.21.15.167:0/3339740892 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa233479700 1 -- 172.21.15.167:0/2865227281 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x565322ad6b00 con 0x5653229dd000 2024-09-13T23:43:26.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa233479700 1 --2- 172.21.15.167:0/2865227281 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322d94000 0x565322cc3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa233479700 1 -- 172.21.15.167:0/2865227281 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x565322a43180 con 0x5653229dd000 2024-09-13T23:43:26.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa236c80700 1 -- 172.21.15.167:0/3339740892 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653229ee000 con 0x5653229dd800 2024-09-13T23:43:26.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa23647f700 1 --2- 172.21.15.167:0/2865227281 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322d94000 0x565322cc3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa236c80700 1 --2- 172.21.15.167:0/3339740892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 0x565322cc2b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x56532384f560 tx=0x565322d3cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac462287b00df61 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa23647f700 1 --2- 172.21.15.167:0/2865227281 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322d94000 0x565322cc3080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x56532384f6e0 tx=0x565322d3c360 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa230c74700 1 -- 172.21.15.167:0/3339740892 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322a818c0 con 0x5653229dd800 2024-09-13T23:43:26.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa230c74700 1 -- 172.21.15.167:0/3339740892 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565322da6000 con 0x5653229dd800 2024-09-13T23:43:26.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.671+0000 7fa230c74700 1 -- 172.21.15.167:0/3339740892 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322a814a0 con 0x5653229dd800 2024-09-13T23:43:26.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 -- 172.21.15.167:0/3339740892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 msgr2=0x565322cc2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/3339740892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 0x565322cc2b00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x56532384f560 tx=0x565322d3cde0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 -- 172.21.15.167:0/3339740892 shutdown_connections 2024-09-13T23:43:26.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/3339740892 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 0x565322cc2b00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 -- 172.21.15.167:0/3339740892 >> 172.21.15.167:0/3339740892 conn(0x5653229dd400 msgr2=0x565322add800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 -- 172.21.15.167:0/3339740892 shutdown_connections 2024-09-13T23:43:26.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 -- 172.21.15.167:0/3339740892 wait complete. 2024-09-13T23:43:26.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 Processor -- start 2024-09-13T23:43:26.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 -- start start 2024-09-13T23:43:26.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 0x565322cc2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x565322a028c0 con 0x5653229dd800 2024-09-13T23:43:26.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa236c80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 0x565322cc2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa236c80700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 0x565322cc2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50632/0 (socket says 172.21.15.167:50632) 2024-09-13T23:43:26.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 learned_addr learned my addr 172.21.15.167:0/936150675 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653229eeea0 con 0x5653229dd800 2024-09-13T23:43:26.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa236c80700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 0x565322cc2b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x565322b39f20 tx=0x565322d3c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa22f471700 1 -- 172.21.15.167:0/936150675 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322a81b80 con 0x5653229dd800 2024-09-13T23:43:26.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa22f471700 1 -- 172.21.15.167:0/936150675 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565322da6180 con 0x5653229dd800 2024-09-13T23:43:26.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5653229ee820 con 0x5653229dd800 2024-09-13T23:43:26.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa22f471700 1 -- 172.21.15.167:0/936150675 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322a818c0 con 0x5653229dd800 2024-09-13T23:43:26.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.675+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653229ee9c0 con 0x5653229dd800 2024-09-13T23:43:26.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa22f471700 1 -- 172.21.15.167:0/936150675 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x565322ad7080 con 0x5653229dd800 2024-09-13T23:43:26.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa22f471700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322d94c00 0x565322cc3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa22f471700 1 -- 172.21.15.167:0/936150675 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x565322a43340 con 0x5653229dd800 2024-09-13T23:43:26.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa235c7e700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322d94c00 0x565322cc3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x565322d95000 0x565322cc3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95400 con 0x565322d95000 2024-09-13T23:43:26.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa236c80700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x565322d95000 0x565322cc3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa235c7e700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322d94c00 0x565322cc3600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x565322cbd920 tx=0x565322ae1800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa236c80700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x565322d95000 0x565322cc3b80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 20 realm 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95000 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95c00 con 0x565322d95000 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x565322d95400 0x565322cc4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.679+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95400 2024-09-13T23:43:26.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa23647f700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x565322d95400 0x565322cc4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa23647f700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x565322d95400 0x565322cc4100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa237cc6f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:26.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa237cc6f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95c00 con 0x565322d95400 2024-09-13T23:43:26.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa237cc6f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:26.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa237cc6f40 20 searching for the correct realm 2024-09-13T23:43:26.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x565322e4c000 0x565322cc4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4c400 con 0x565322e4c000 2024-09-13T23:43:26.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa235c7e700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x565322e4c000 0x565322cc4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa235c7e700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x565322e4c000 0x565322cc4680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.683+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4c400 con 0x565322d95400 2024-09-13T23:43:26.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa23647f700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x565322e4c400 0x565322cc4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c400 2024-09-13T23:43:26.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa236c80700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x565322e4c400 0x565322cc4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa236c80700 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x565322e4c400 0x565322cc4c00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95000 2024-09-13T23:43:26.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95000 2024-09-13T23:43:26.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c000 2024-09-13T23:43:26.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95000 2024-09-13T23:43:26.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.687+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95400 2024-09-13T23:43:26.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95400 2024-09-13T23:43:26.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95000 2024-09-13T23:43:26.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c400 2024-09-13T23:43:26.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95000 2024-09-13T23:43:26.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c400 2024-09-13T23:43:26.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95400 2024-09-13T23:43:26.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.691+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c000 2024-09-13T23:43:26.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95000 2024-09-13T23:43:26.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95400 2024-09-13T23:43:26.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c000 2024-09-13T23:43:26.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95000 2024-09-13T23:43:26.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95000 2024-09-13T23:43:26.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95000 2024-09-13T23:43:26.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.695+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c000 2024-09-13T23:43:26.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c400 2024-09-13T23:43:26.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95000 2024-09-13T23:43:26.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c400 2024-09-13T23:43:26.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c000 2024-09-13T23:43:26.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c400 2024-09-13T23:43:26.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95400 2024-09-13T23:43:26.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.699+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95400 2024-09-13T23:43:26.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c000 2024-09-13T23:43:26.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95400 2024-09-13T23:43:26.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95400 2024-09-13T23:43:26.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa237cc6f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:26.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa237cc6f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:26.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa237cc6f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:26.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa237cc6f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:26.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa237cc6f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4c800 con 0x565322d95000 2024-09-13T23:43:26.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa237cc6f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa237cc6f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4cc00 con 0x565322d95400 2024-09-13T23:43:26.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.703+0000 7fa237cc6f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:26.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4d000 con 0x565322d95400 2024-09-13T23:43:26.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:26.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 20 zone default found 2024-09-13T23:43:26.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 4 Realm: () 2024-09-13T23:43:26.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:26.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:26.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:26.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 20 zonegroup default 2024-09-13T23:43:26.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4d400 con 0x565322e4c400 2024-09-13T23:43:26.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4d800 con 0x565322e4c000 2024-09-13T23:43:26.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4dc00 con 0x565322e4c000 2024-09-13T23:43:26.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 20 started sync module instance, tier type = 2024-09-13T23:43:26.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:26.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.707+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4d800 con 0x565322d95400 2024-09-13T23:43:26.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.711+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.711+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94915066043136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4dc00 con 0x565322d95400 2024-09-13T23:43:26.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.711+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4cc00 con 0x565322d95000 2024-09-13T23:43:26.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.715+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.715+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94915066044288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4d000 con 0x565322d95000 2024-09-13T23:43:26.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.715+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4d800 con 0x565322d95400 2024-09-13T23:43:26.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.715+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94915066043136] v33'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.715+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94915066044288] v33'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.715+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.715+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94915066045440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95c00 con 0x565322d95400 2024-09-13T23:43:26.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.715+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c400 2024-09-13T23:43:26.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.719+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.719+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94915066046592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4c800 con 0x565322e4c400 2024-09-13T23:43:26.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.719+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4d400 con 0x565322d95400 2024-09-13T23:43:26.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.719+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94915066045440] v33'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.719+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94915066046592] v33'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.723+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.723+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94915066047744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4cc00 con 0x565322d95400 2024-09-13T23:43:26.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.723+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4d000 con 0x565322e4c000 2024-09-13T23:43:26.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.723+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.723+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94915066048896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e58000 con 0x565322e4c000 2024-09-13T23:43:26.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.723+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e58400 con 0x565322e4c000 2024-09-13T23:43:26.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.723+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94915066047744] v33'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94915066048896] v33'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94915066050048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e58800 con 0x565322e4c000 2024-09-13T23:43:26.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e58c00 con 0x565322e4c400 2024-09-13T23:43:26.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'33 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94915066051200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e59000 con 0x565322e4c400 2024-09-13T23:43:26.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa237cc6f40 20 add_watcher() i=0 2024-09-13T23:43:26.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa237cc6f40 20 add_watcher() i=1 2024-09-13T23:43:26.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa237cc6f40 20 add_watcher() i=2 2024-09-13T23:43:26.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa237cc6f40 20 add_watcher() i=3 2024-09-13T23:43:26.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa237cc6f40 20 add_watcher() i=4 2024-09-13T23:43:26.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa237cc6f40 20 add_watcher() i=5 2024-09-13T23:43:26.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94915066050048] v33'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.727+0000 7fa237cc6f40 20 add_watcher() i=6 2024-09-13T23:43:26.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.731+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94915066051200] v33'34 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.731+0000 7fa237cc6f40 20 add_watcher() i=7 2024-09-13T23:43:26.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.731+0000 7fa237cc6f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:26.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.731+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e59400 con 0x565322d95000 2024-09-13T23:43:26.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.731+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.731+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94915067764736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e59800 con 0x565322d95000 2024-09-13T23:43:26.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.731+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94915067764736] v33'56 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.731+0000 7fa21d44d700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:26.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.731+0000 7fa237cc6f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:26.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.731+0000 7fa237cc6f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:26.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.731+0000 7fa237cc6f40 1 Processor -- start 2024-09-13T23:43:26.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.731+0000 7fa237cc6f40 1 -- start start 2024-09-13T23:43:26.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 0x565322cc5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x565322a6a3c0 con 0x565322e58c00 2024-09-13T23:43:26.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa235c7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 0x565322cc5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa235c7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 0x565322cc5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50638/0 (socket says 172.21.15.167:50638) 2024-09-13T23:43:26.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa235c7e700 1 -- 172.21.15.167:0/1826262231 learned_addr learned my addr 172.21.15.167:0/1826262231 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa235c7e700 1 -- 172.21.15.167:0/1826262231 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653229eeb60 con 0x565322e58c00 2024-09-13T23:43:26.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa235c7e700 1 --2- 172.21.15.167:0/1826262231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 0x565322cc5180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x565322b380c0 tx=0x565322ae1860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c9d10b484fa6cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa21cc4c700 1 -- 172.21.15.167:0/1826262231 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322a81760 con 0x565322e58c00 2024-09-13T23:43:26.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa21cc4c700 1 -- 172.21.15.167:0/1826262231 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565322da6600 con 0x565322e58c00 2024-09-13T23:43:26.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa21cc4c700 1 -- 172.21.15.167:0/1826262231 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322e70000 con 0x565322e58c00 2024-09-13T23:43:26.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1826262231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 msgr2=0x565322cc5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/1826262231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 0x565322cc5180 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x565322b380c0 tx=0x565322ae1860 comp rx=0 tx=0).stop 2024-09-13T23:43:26.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1826262231 shutdown_connections 2024-09-13T23:43:26.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/1826262231 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 0x565322cc5180 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1826262231 >> 172.21.15.167:0/1826262231 conn(0x565322e59c00 msgr2=0x565322ade000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1826262231 shutdown_connections 2024-09-13T23:43:26.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1826262231 wait complete. 2024-09-13T23:43:26.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 Processor -- start 2024-09-13T23:43:26.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 -- start start 2024-09-13T23:43:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 0x565322cc5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.735+0000 7fa237cc6f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x565322a6a500 con 0x565322e58c00 2024-09-13T23:43:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa235c7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 0x565322cc5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa235c7e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 0x565322cc5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50640/0 (socket says 172.21.15.167:50640) 2024-09-13T23:43:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa235c7e700 1 -- 172.21.15.167:0/1908899694 learned_addr learned my addr 172.21.15.167:0/1908899694 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa235c7e700 1 -- 172.21.15.167:0/1908899694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653229eed00 con 0x565322e58c00 2024-09-13T23:43:26.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa235c7e700 1 --2- 172.21.15.167:0/1908899694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 0x565322cc5180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x565322d75620 tx=0x565322e68300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa21b449700 1 -- 172.21.15.167:0/1908899694 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322e70160 con 0x565322e58c00 2024-09-13T23:43:26.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa21b449700 1 -- 172.21.15.167:0/1908899694 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x565322da6300 con 0x565322e58c00 2024-09-13T23:43:26.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1908899694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5653229ee340 con 0x565322e58c00 2024-09-13T23:43:26.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa21b449700 1 -- 172.21.15.167:0/1908899694 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x565322e702c0 con 0x565322e58c00 2024-09-13T23:43:26.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1908899694 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653229efd40 con 0x565322e58c00 2024-09-13T23:43:26.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa21b449700 1 -- 172.21.15.167:0/1908899694 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x565322ad7b80 con 0x565322e58c00 2024-09-13T23:43:26.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa21b449700 1 --2- 172.21.15.167:0/1908899694 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322ee6000 0x565322eec000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa21b449700 1 -- 172.21.15.167:0/1908899694 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x565322a43500 con 0x565322e58c00 2024-09-13T23:43:26.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa236c80700 1 --2- 172.21.15.167:0/1908899694 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322ee6000 0x565322eec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.739+0000 7fa237cc6f40 5 note: GC not initialized 2024-09-13T23:43:26.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa236c80700 1 --2- 172.21.15.167:0/1908899694 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322ee6000 0x565322eec000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x565322de7d40 tx=0x565322d3d7a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa218c44700 20 reqs_thread_entry: start 2024-09-13T23:43:26.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa237cc6f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa237cc6f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa216c40700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:56 2024 2024-09-13T23:43:26.752 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:26.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa237cc6f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa237cc6f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e58000 con 0x565322d95000 2024-09-13T23:43:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa21641e700 20 reqs_thread_entry: start 2024-09-13T23:43:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa237cc6f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:26.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa237cc6f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa237cc6f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa237cc6f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e58400 con 0x565322e4c400 2024-09-13T23:43:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.743+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4d000 con 0x565322d95000 2024-09-13T23:43:26.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.747+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~357] v33'2 uv2 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.747+0000 7fa237cc6f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:26.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.747+0000 7fa237cc6f40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:26.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.747+0000 7fa237cc6f40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:26.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.747+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94915067767040 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322d95400 2024-09-13T23:43:26.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.747+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94915066047744 notify 141733920769 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x565322d4e820 con 0x565322d95400 2024-09-13T23:43:26.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.747+0000 7fa230c74700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920769 cookie 94915066047744 notifier 4318 bl.length()=533 2024-09-13T23:43:26.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.747+0000 7fa230c74700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:26.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.747+0000 7fa230c74700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:26.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.747+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 notify.4 [notify cookie 94915067767040 out=8b] v0'0 uv33 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.747+0000 7fa230c74700 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322e4dc00 con 0x565322d95400 2024-09-13T23:43:26.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.747+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.751+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94915067767040 notify 141733920769 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x565322d4e9c0 con 0x565322d95400 2024-09-13T23:43:26.759 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:26.759 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:26.759 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:26.759 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:26.759 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 1, 2024-09-13T23:43:26.759 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:26.759 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:26.759 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:26.759 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:26.759 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:26.759 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:26.760 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:26.761 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.751+0000 7fa21d44d700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.751+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94915067764736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322ee6400 con 0x565322d95000 2024-09-13T23:43:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.751+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94915067764736] v33'57 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.751+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94915066043136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322ee6800 con 0x565322d95400 2024-09-13T23:43:26.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.755+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94915066043136] v33'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.755+0000 7fa237cc6f40 20 remove_watcher() i=0 2024-09-13T23:43:26.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.755+0000 7fa237cc6f40 2 removed watcher, disabling cache 2024-09-13T23:43:26.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.755+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94915066044288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322ee6c00 con 0x565322d95000 2024-09-13T23:43:26.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.759+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94915066044288] v33'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322d95000 2024-09-13T23:43:26.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.759+0000 7fa237cc6f40 20 remove_watcher() i=1 2024-09-13T23:43:26.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.759+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94915066045440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322ee7000 con 0x565322d95400 2024-09-13T23:43:26.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.759+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94915066045440] v33'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.759+0000 7fa237cc6f40 20 remove_watcher() i=2 2024-09-13T23:43:26.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.759+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94915066046592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322ee7400 con 0x565322e4c400 2024-09-13T23:43:26.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.763+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94915066046592] v33'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.763+0000 7fa237cc6f40 20 remove_watcher() i=3 2024-09-13T23:43:26.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.763+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94915066047744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322ee7800 con 0x565322d95400 2024-09-13T23:43:26.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.767+0000 7fa23647f700 1 -- 172.21.15.167:0/936150675 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94915066047744] v33'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43200 con 0x565322d95400 2024-09-13T23:43:26.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.767+0000 7fa237cc6f40 20 remove_watcher() i=4 2024-09-13T23:43:26.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.767+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94915066048896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322ee7c00 con 0x565322e4c000 2024-09-13T23:43:26.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.767+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94915066048896] v33'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.767+0000 7fa237cc6f40 20 remove_watcher() i=5 2024-09-13T23:43:26.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.767+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94915066050048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95c00 con 0x565322e4c000 2024-09-13T23:43:26.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94915066050048] v33'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e43440 con 0x565322e4c000 2024-09-13T23:43:26.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 20 remove_watcher() i=6 2024-09-13T23:43:26.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94915066051200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x565322d95800 con 0x565322e4c400 2024-09-13T23:43:26.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa236c80700 1 -- 172.21.15.167:0/936150675 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94915066051200] v33'35 uv33 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x565322e42b40 con 0x565322e4c400 2024-09-13T23:43:26.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 20 remove_watcher() i=7 2024-09-13T23:43:26.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x565322d95400 msgr2=0x565322cc4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x565322d95400 0x565322cc4100 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x565322d95000 msgr2=0x565322cc3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x565322d95000 0x565322cc3b80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x565322e4c400 msgr2=0x565322cc4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x565322e4c400 0x565322cc4c00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x565322e4c000 msgr2=0x565322cc4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x565322e4c000 0x565322cc4680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322d94c00 msgr2=0x565322cc3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322d94c00 0x565322cc3600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x565322cbd920 tx=0x565322ae1800 comp rx=0 tx=0).stop 2024-09-13T23:43:26.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 msgr2=0x565322cc2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa235c7e700 1 -- 172.21.15.167:0/936150675 reap_dead start 2024-09-13T23:43:26.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.771+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 0x565322cc2b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x565322b39f20 tx=0x565322d3c000 comp rx=0 tx=0).stop 2024-09-13T23:43:26.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 shutdown_connections 2024-09-13T23:43:26.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/936150675 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd800 0x565322cc2b00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 >> 172.21.15.167:0/936150675 conn(0x5653229dd400 msgr2=0x565322add800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 shutdown_connections 2024-09-13T23:43:26.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/936150675 wait complete. 2024-09-13T23:43:26.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1908899694 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322ee6000 msgr2=0x565322eec000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/1908899694 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322ee6000 0x565322eec000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x565322de7d40 tx=0x565322d3d7a0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1908899694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 msgr2=0x565322cc5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/1908899694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 0x565322cc5180 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x565322d75620 tx=0x565322e68300 comp rx=0 tx=0).stop 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1908899694 shutdown_connections 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/1908899694 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322ee6000 0x565322eec000 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/1908899694 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x565322e58c00 0x565322cc5180 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1908899694 >> 172.21.15.167:0/1908899694 conn(0x565322e59c00 msgr2=0x565322ade000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1908899694 shutdown_connections 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/1908899694 wait complete. 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/2865227281 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322d94000 msgr2=0x565322cc3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/2865227281 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322d94000 0x565322cc3080 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x56532384f6e0 tx=0x565322d3c360 comp rx=0 tx=0).stop 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/2865227281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 msgr2=0x565322cc2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/2865227281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 0x565322cc2580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x56532384e0c0 tx=0x565322ae1a40 comp rx=0 tx=0).stop 2024-09-13T23:43:26.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/2865227281 shutdown_connections 2024-09-13T23:43:26.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/2865227281 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x565322d94000 0x565322cc3080 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 --2- 172.21.15.167:0/2865227281 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5653229dd000 0x565322cc2580 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/2865227281 >> 172.21.15.167:0/2865227281 conn(0x5653229dcc00 msgr2=0x565322add000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/2865227281 shutdown_connections 2024-09-13T23:43:26.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.775+0000 7fa237cc6f40 1 -- 172.21.15.167:0/2865227281 wait complete. 2024-09-13T23:43:26.791 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:26.791 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-13T23:43:26.791 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-13T23:43:26.792 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-13T23:43:26.833 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:26.833 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:26.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.823+0000 7f6bf060ef40 1 Processor -- start 2024-09-13T23:43:26.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.823+0000 7f6bf060ef40 1 -- start start 2024-09-13T23:43:26.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.823+0000 7f6bf060ef40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55977fc41400 0x55977ff26580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.823+0000 7f6bf060ef40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55977fd05500 con 0x55977fc41000 2024-09-13T23:43:26.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.823+0000 7f6bf060ef40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55977fccea00 con 0x55977fc41400 2024-09-13T23:43:26.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.823+0000 7f6bee5c6700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55977fc41400 0x55977ff26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.823+0000 7f6beedc7700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55977fc41000 0x55977fd40800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45238/0 (socket says 172.21.15.167:45238) 2024-09-13T23:43:26.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6beedc7700 1 -- 172.21.15.167:0/1620795626 learned_addr learned my addr 172.21.15.167:0/1620795626 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:26.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bee5c6700 1 -- 172.21.15.167:0/1620795626 >> v1:172.21.15.12:6789/0 conn(0x55977fc41000 legacy=0x55977fd40800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bee5c6700 1 -- 172.21.15.167:0/1620795626 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55977fc53d40 con 0x55977fc41400 2024-09-13T23:43:26.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6beddc5700 1 -- 172.21.15.167:0/1620795626 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997966550 0 0) 0x55977fd9c000 con 0x55977fc41000 2024-09-13T23:43:26.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bee5c6700 1 --2- 172.21.15.167:0/1620795626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41400 0x55977ff26580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55977ff21920 tx=0x55977fd45920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e067bed3f572b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6beddc5700 1 -- 172.21.15.167:0/1620795626 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55977fce54a0 con 0x55977fc41400 2024-09-13T23:43:26.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6beddc5700 1 -- 172.21.15.167:0/1620795626 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55977fd05800 con 0x55977fc41400 2024-09-13T23:43:26.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6beddc5700 1 -- 172.21.15.167:0/1620795626 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55977fce5600 con 0x55977fc41400 2024-09-13T23:43:26.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bf060ef40 1 -- 172.21.15.167:0/1620795626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41400 msgr2=0x55977ff26580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/1620795626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41400 0x55977ff26580 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55977ff21920 tx=0x55977fd45920 comp rx=0 tx=0).stop 2024-09-13T23:43:26.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bf060ef40 1 -- 172.21.15.167:0/1620795626 shutdown_connections 2024-09-13T23:43:26.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/1620795626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41400 0x55977ff26580 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bf060ef40 1 -- 172.21.15.167:0/1620795626 >> 172.21.15.167:0/1620795626 conn(0x55977fc40c00 msgr2=0x55977fd40000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bf060ef40 1 -- 172.21.15.167:0/1620795626 shutdown_connections 2024-09-13T23:43:26.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bf060ef40 1 -- 172.21.15.167:0/1620795626 wait complete. 2024-09-13T23:43:26.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bf060ef40 1 Processor -- start 2024-09-13T23:43:26.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bf060ef40 1 -- start start 2024-09-13T23:43:26.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bf060ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 0x55977ff26580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.827+0000 7f6bf060ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fcce500 con 0x55977fc41000 2024-09-13T23:43:26.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6beedc7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 0x55977ff26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6beedc7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 0x55977ff26580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50654/0 (socket says 172.21.15.167:50654) 2024-09-13T23:43:26.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6beedc7700 1 -- 172.21.15.167:0/1947323828 learned_addr learned my addr 172.21.15.167:0/1947323828 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6beedc7700 1 -- 172.21.15.167:0/1947323828 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55977fc53ba0 con 0x55977fc41000 2024-09-13T23:43:26.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6beedc7700 1 --2- 172.21.15.167:0/1947323828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 0x55977ff26580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x559780a0bd70 tx=0x55977fd45b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e83749856aa8101 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6bed5c4700 1 -- 172.21.15.167:0/1947323828 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55977fce54a0 con 0x55977fc41000 2024-09-13T23:43:26.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6bed5c4700 1 -- 172.21.15.167:0/1947323828 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55977fd05b00 con 0x55977fc41000 2024-09-13T23:43:26.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6bed5c4700 1 -- 172.21.15.167:0/1947323828 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55977fce5600 con 0x55977fc41000 2024-09-13T23:43:26.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6bf060ef40 1 -- 172.21.15.167:0/1947323828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 msgr2=0x55977ff26580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/1947323828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 0x55977ff26580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x559780a0bd70 tx=0x55977fd45b60 comp rx=0 tx=0).stop 2024-09-13T23:43:26.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6bf060ef40 1 -- 172.21.15.167:0/1947323828 shutdown_connections 2024-09-13T23:43:26.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/1947323828 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 0x55977ff26580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6bf060ef40 1 -- 172.21.15.167:0/1947323828 >> 172.21.15.167:0/1947323828 conn(0x55977fc40c00 msgr2=0x55977fd41000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6bf060ef40 1 -- 172.21.15.167:0/1947323828 shutdown_connections 2024-09-13T23:43:26.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6bf060ef40 1 -- 172.21.15.167:0/1947323828 wait complete. 2024-09-13T23:43:26.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6bf060ef40 1 Processor -- start 2024-09-13T23:43:26.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.831+0000 7f6bf060ef40 1 -- start start 2024-09-13T23:43:26.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6bf060ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 0x55977ff26580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6bf060ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fcce3c0 con 0x55977fc41000 2024-09-13T23:43:26.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6beedc7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 0x55977ff26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6beedc7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 0x55977ff26580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50656/0 (socket says 172.21.15.167:50656) 2024-09-13T23:43:26.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6beedc7700 1 -- 172.21.15.167:0/4281753315 learned_addr learned my addr 172.21.15.167:0/4281753315 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6beedc7700 1 -- 172.21.15.167:0/4281753315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55977fc53a00 con 0x55977fc41000 2024-09-13T23:43:26.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6beedc7700 1 --2- 172.21.15.167:0/4281753315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 0x55977ff26580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x559780a0be60 tx=0x55977fd45a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6bebdc1700 1 -- 172.21.15.167:0/4281753315 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55977fce58c0 con 0x55977fc41000 2024-09-13T23:43:26.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6bebdc1700 1 -- 172.21.15.167:0/4281753315 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55977fd05c80 con 0x55977fc41000 2024-09-13T23:43:26.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6bf060ef40 1 -- 172.21.15.167:0/4281753315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55977fc53860 con 0x55977fc41000 2024-09-13T23:43:26.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6bebdc1700 1 -- 172.21.15.167:0/4281753315 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55977fce54a0 con 0x55977fc41000 2024-09-13T23:43:26.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6bf060ef40 1 -- 172.21.15.167:0/4281753315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55977fc536c0 con 0x55977fc41000 2024-09-13T23:43:26.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6bf060ef40 1 Processor -- start 2024-09-13T23:43:26.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6bf060ef40 1 -- start start 2024-09-13T23:43:26.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6bf060ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 0x55977ff26b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6bf060ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fcce000 con 0x55977fc41800 2024-09-13T23:43:26.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.835+0000 7f6bebdc1700 1 -- 172.21.15.167:0/4281753315 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55977fd3ab00 con 0x55977fc41000 2024-09-13T23:43:26.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bef5c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 0x55977ff26b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bebdc1700 1 --2- 172.21.15.167:0/4281753315 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55977ffec000 0x55977ff27080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bef5c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 0x55977ff26b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50660/0 (socket says 172.21.15.167:50660) 2024-09-13T23:43:26.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bef5c8700 1 -- 172.21.15.167:0/156686284 learned_addr learned my addr 172.21.15.167:0/156686284 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bebdc1700 1 -- 172.21.15.167:0/4281753315 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55977fca7180 con 0x55977fc41000 2024-09-13T23:43:26.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6beedc7700 1 --2- 172.21.15.167:0/4281753315 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55977ffec000 0x55977ff27080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bef5c8700 1 -- 172.21.15.167:0/156686284 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55977fc52000 con 0x55977fc41800 2024-09-13T23:43:26.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bef5c8700 1 --2- 172.21.15.167:0/156686284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 0x55977ff26b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55977ffca510 tx=0x55977ffa0de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7f9598106f09f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6beedc7700 1 --2- 172.21.15.167:0/4281753315 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55977ffec000 0x55977ff27080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55977ff6d4d0 tx=0x55977ffa0240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6be95bc700 1 -- 172.21.15.167:0/156686284 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55977fce58c0 con 0x55977fc41800 2024-09-13T23:43:26.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6be95bc700 1 -- 172.21.15.167:0/156686284 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55977fff2000 con 0x55977fc41800 2024-09-13T23:43:26.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6be95bc700 1 -- 172.21.15.167:0/156686284 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55977fce54a0 con 0x55977fc41800 2024-09-13T23:43:26.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bf060ef40 1 -- 172.21.15.167:0/156686284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 msgr2=0x55977ff26b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/156686284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 0x55977ff26b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x55977ffca510 tx=0x55977ffa0de0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bf060ef40 1 -- 172.21.15.167:0/156686284 shutdown_connections 2024-09-13T23:43:26.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/156686284 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 0x55977ff26b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bf060ef40 1 -- 172.21.15.167:0/156686284 >> 172.21.15.167:0/156686284 conn(0x55977fc41400 msgr2=0x55977fd41800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bf060ef40 1 -- 172.21.15.167:0/156686284 shutdown_connections 2024-09-13T23:43:26.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bf060ef40 1 -- 172.21.15.167:0/156686284 wait complete. 2024-09-13T23:43:26.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bf060ef40 1 Processor -- start 2024-09-13T23:43:26.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bf060ef40 1 -- start start 2024-09-13T23:43:26.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bf060ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 0x55977ff26b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.839+0000 7f6bf060ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fc66640 con 0x55977fc41800 2024-09-13T23:43:26.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6bef5c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 0x55977ff26b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6bef5c8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 0x55977ff26b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50674/0 (socket says 172.21.15.167:50674) 2024-09-13T23:43:26.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 learned_addr learned my addr 172.21.15.167:0/3738950467 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55977fc52ea0 con 0x55977fc41800 2024-09-13T23:43:26.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6bef5c8700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 0x55977ff26b00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x559780a0bf80 tx=0x55977ffa0840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6be7db9700 1 -- 172.21.15.167:0/3738950467 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55977fce5b80 con 0x55977fc41800 2024-09-13T23:43:26.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6be7db9700 1 -- 172.21.15.167:0/3738950467 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55977fff2180 con 0x55977fc41800 2024-09-13T23:43:26.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55977fc52820 con 0x55977fc41800 2024-09-13T23:43:26.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6be7db9700 1 -- 172.21.15.167:0/3738950467 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55977fce58c0 con 0x55977fc41800 2024-09-13T23:43:26.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55977fc529c0 con 0x55977fc41800 2024-09-13T23:43:26.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6be7db9700 1 -- 172.21.15.167:0/3738950467 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55977fd3adc0 con 0x55977fc41800 2024-09-13T23:43:26.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6be7db9700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55977ffecc00 0x55977ff27600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6be7db9700 1 -- 172.21.15.167:0/3738950467 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55977fca7340 con 0x55977fc41800 2024-09-13T23:43:26.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.843+0000 7f6bee5c6700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55977ffecc00 0x55977ff27600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55977ffed000 0x55977ff27b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed400 con 0x55977ffed000 2024-09-13T23:43:26.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bef5c8700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55977ffed000 0x55977ff27b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bee5c6700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55977ffecc00 0x55977ff27600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55977ff21920 tx=0x55977fd45800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bef5c8700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55977ffed000 0x55977ff27b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 20 realm 2024-09-13T23:43:26.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed000 2024-09-13T23:43:26.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:26.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffedc00 con 0x55977ffed000 2024-09-13T23:43:26.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55977ffed400 0x55977ff28100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed400 2024-09-13T23:43:26.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6beedc7700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55977ffed400 0x55977ff28100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.847+0000 7f6beedc7700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55977ffed400 0x55977ff28100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bf060ef40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:26.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bf060ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffedc00 con 0x55977ffed400 2024-09-13T23:43:26.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bf060ef40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:26.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bf060ef40 20 searching for the correct realm 2024-09-13T23:43:26.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55977ff82000 0x55977ff28680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff82400 con 0x55977ff82000 2024-09-13T23:43:26.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bee5c6700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55977ff82000 0x55977ff28680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bee5c6700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55977ff82000 0x55977ff28680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff82400 con 0x55977ffed400 2024-09-13T23:43:26.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6beedc7700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55977ff82400 0x55977ff28c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82400 2024-09-13T23:43:26.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bef5c8700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55977ff82400 0x55977ff28c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.851+0000 7f6bef5c8700 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55977ff82400 0x55977ff28c00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed000 2024-09-13T23:43:26.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed000 2024-09-13T23:43:26.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82000 2024-09-13T23:43:26.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed000 2024-09-13T23:43:26.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed400 2024-09-13T23:43:26.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed400 2024-09-13T23:43:26.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.855+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed000 2024-09-13T23:43:26.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82400 2024-09-13T23:43:26.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed000 2024-09-13T23:43:26.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82400 2024-09-13T23:43:26.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed400 2024-09-13T23:43:26.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82000 2024-09-13T23:43:26.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed000 2024-09-13T23:43:26.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.859+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed400 2024-09-13T23:43:26.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82000 2024-09-13T23:43:26.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed000 2024-09-13T23:43:26.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed000 2024-09-13T23:43:26.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed000 2024-09-13T23:43:26.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82000 2024-09-13T23:43:26.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82400 2024-09-13T23:43:26.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.863+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed000 2024-09-13T23:43:26.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.867+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.867+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82400 2024-09-13T23:43:26.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.867+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.867+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82000 2024-09-13T23:43:26.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.867+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.867+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82400 2024-09-13T23:43:26.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.867+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.867+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed400 2024-09-13T23:43:26.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.867+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.867+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed400 2024-09-13T23:43:26.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.867+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.867+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82000 2024-09-13T23:43:26.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed400 2024-09-13T23:43:26.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed400 2024-09-13T23:43:26.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bf060ef40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:26.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bf060ef40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:26.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bf060ef40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:26.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bf060ef40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:26.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bf060ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff82800 con 0x55977ffed000 2024-09-13T23:43:26.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bf060ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bf060ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff82c00 con 0x55977ffed400 2024-09-13T23:43:26.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bf060ef40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:26.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bf060ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.871+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff83000 con 0x55977ffed400 2024-09-13T23:43:26.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:26.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 20 zone default found 2024-09-13T23:43:26.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 4 Realm: () 2024-09-13T23:43:26.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:26.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:26.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:26.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 20 zonegroup default 2024-09-13T23:43:26.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff83400 con 0x55977ff82400 2024-09-13T23:43:26.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff83800 con 0x55977ff82000 2024-09-13T23:43:26.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff83c00 con 0x55977ff82000 2024-09-13T23:43:26.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:26.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 20 started sync module instance, tier type = 2024-09-13T23:43:26.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:26.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.875+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff83800 con 0x55977ffed400 2024-09-13T23:43:26.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.879+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.879+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94109174979328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff83c00 con 0x55977ffed400 2024-09-13T23:43:26.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.879+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff82c00 con 0x55977ffed000 2024-09-13T23:43:26.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.879+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94109174979328] v33'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.879+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.879+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94109174980480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff83000 con 0x55977ffed000 2024-09-13T23:43:26.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.883+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff83800 con 0x55977ffed400 2024-09-13T23:43:26.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.883+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.883+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94109174980480] v33'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.883+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94109174981632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffedc00 con 0x55977ffed400 2024-09-13T23:43:26.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.883+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ff82400 2024-09-13T23:43:26.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.887+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.887+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94109174982784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff82800 con 0x55977ff82400 2024-09-13T23:43:26.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.887+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff83400 con 0x55977ffed400 2024-09-13T23:43:26.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.887+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94109174981632] v33'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.887+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94109174982784] v33'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.891+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.891+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94109174983936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff82c00 con 0x55977ffed400 2024-09-13T23:43:26.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.891+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff83000 con 0x55977ff82000 2024-09-13T23:43:26.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.891+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.891+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94109174985088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5597800b0000 con 0x55977ff82000 2024-09-13T23:43:26.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.891+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5597800b0400 con 0x55977ff82000 2024-09-13T23:43:26.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.891+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94109174983936] v33'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94109174986240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5597800b0800 con 0x55977ff82000 2024-09-13T23:43:26.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5597800b0c00 con 0x55977ff82400 2024-09-13T23:43:26.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94109174985088] v33'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94109174986240] v33'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'36 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94109174987392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5597800b1000 con 0x55977ff82400 2024-09-13T23:43:26.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bf060ef40 20 add_watcher() i=0 2024-09-13T23:43:26.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bf060ef40 20 add_watcher() i=1 2024-09-13T23:43:26.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bf060ef40 20 add_watcher() i=2 2024-09-13T23:43:26.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bf060ef40 20 add_watcher() i=3 2024-09-13T23:43:26.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bf060ef40 20 add_watcher() i=4 2024-09-13T23:43:26.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bf060ef40 20 add_watcher() i=5 2024-09-13T23:43:26.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bf060ef40 20 add_watcher() i=6 2024-09-13T23:43:26.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.895+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94109174987392] v33'37 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.899+0000 7f6bf060ef40 20 add_watcher() i=7 2024-09-13T23:43:26.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.899+0000 7f6bf060ef40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:26.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.899+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5597800b1400 con 0x55977ffed000 2024-09-13T23:43:26.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.899+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.899+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94109176651776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5597800b1800 con 0x55977ffed000 2024-09-13T23:43:26.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.899+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94109176651776] v33'58 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.899+0000 7f6bd5d95700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:26.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.899+0000 7f6bf060ef40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:26.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.899+0000 7f6bf060ef40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:26.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.899+0000 7f6bf060ef40 1 Processor -- start 2024-09-13T23:43:26.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.899+0000 7f6bf060ef40 1 -- start start 2024-09-13T23:43:26.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 0x55977ff29180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fc66780 con 0x5597800b0c00 2024-09-13T23:43:26.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bee5c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 0x55977ff29180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bee5c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 0x55977ff29180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50686/0 (socket says 172.21.15.167:50686) 2024-09-13T23:43:26.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bee5c6700 1 -- 172.21.15.167:0/2146555288 learned_addr learned my addr 172.21.15.167:0/2146555288 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bee5c6700 1 -- 172.21.15.167:0/2146555288 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55977fc52b60 con 0x5597800b0c00 2024-09-13T23:43:26.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bee5c6700 1 --2- 172.21.15.167:0/2146555288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 0x55977ff29180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55978003b6b0 tx=0x55977fd45860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4449bb11785157 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bd5594700 1 -- 172.21.15.167:0/2146555288 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55977fce5760 con 0x5597800b0c00 2024-09-13T23:43:26.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bd5594700 1 -- 172.21.15.167:0/2146555288 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55977fff2600 con 0x5597800b0c00 2024-09-13T23:43:26.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bd5594700 1 -- 172.21.15.167:0/2146555288 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5597800c8000 con 0x5597800b0c00 2024-09-13T23:43:26.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 -- 172.21.15.167:0/2146555288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 msgr2=0x55977ff29180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/2146555288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 0x55977ff29180 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x55978003b6b0 tx=0x55977fd45860 comp rx=0 tx=0).stop 2024-09-13T23:43:26.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 -- 172.21.15.167:0/2146555288 shutdown_connections 2024-09-13T23:43:26.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/2146555288 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 0x55977ff29180 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 -- 172.21.15.167:0/2146555288 >> 172.21.15.167:0/2146555288 conn(0x5597800b1c00 msgr2=0x55977fd42000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 -- 172.21.15.167:0/2146555288 shutdown_connections 2024-09-13T23:43:26.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 -- 172.21.15.167:0/2146555288 wait complete. 2024-09-13T23:43:26.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 Processor -- start 2024-09-13T23:43:26.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 -- start start 2024-09-13T23:43:26.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 0x55977ff29180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.903+0000 7f6bf060ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977fcce000 con 0x5597800b0c00 2024-09-13T23:43:26.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bee5c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 0x55977ff29180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bee5c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 0x55977ff29180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50696/0 (socket says 172.21.15.167:50696) 2024-09-13T23:43:26.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3098043923 learned_addr learned my addr 172.21.15.167:0/3098043923 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3098043923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55977fc52d00 con 0x5597800b0c00 2024-09-13T23:43:26.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bee5c6700 1 --2- 172.21.15.167:0/3098043923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 0x55977ff29180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5597800be1b0 tx=0x5597800c0300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bd3d91700 1 -- 172.21.15.167:0/3098043923 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5597800c8160 con 0x5597800b0c00 2024-09-13T23:43:26.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bd3d91700 1 -- 172.21.15.167:0/3098043923 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55977fff2300 con 0x5597800b0c00 2024-09-13T23:43:26.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bd3d91700 1 -- 172.21.15.167:0/3098043923 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5597800c82c0 con 0x5597800b0c00 2024-09-13T23:43:26.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3098043923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55977fc52340 con 0x5597800b0c00 2024-09-13T23:43:26.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3098043923 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55977fc53d40 con 0x5597800b0c00 2024-09-13T23:43:26.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bd3d91700 1 -- 172.21.15.167:0/3098043923 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55977fd3bb80 con 0x5597800b0c00 2024-09-13T23:43:26.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bd3d91700 1 --2- 172.21.15.167:0/3098043923 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55978013e000 0x559780144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bd3d91700 1 -- 172.21.15.167:0/3098043923 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55977fca7500 con 0x5597800b0c00 2024-09-13T23:43:26.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bf060ef40 5 note: GC not initialized 2024-09-13T23:43:26.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bef5c8700 1 --2- 172.21.15.167:0/3098043923 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55978013e000 0x559780144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.907+0000 7f6bef5c8700 1 --2- 172.21.15.167:0/3098043923 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55978013e000 0x559780144000 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55978003bcb0 tx=0x55977ffa17a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.911+0000 7f6bd158c700 20 reqs_thread_entry: start 2024-09-13T23:43:26.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.911+0000 7f6bf060ef40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:26.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.911+0000 7f6bf060ef40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:26.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.911+0000 7f6bcf588700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:57 2024 2024-09-13T23:43:26.920 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:26.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.911+0000 7f6bced66700 20 reqs_thread_entry: start 2024-09-13T23:43:26.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.911+0000 7f6bf060ef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:26.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.911+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5597800b0000 con 0x55977ffed000 2024-09-13T23:43:26.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.911+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.911+0000 7f6bf060ef40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 1, 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:26.922 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:26.923 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:26.924 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:26.924 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:26.924 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:26.924 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:26.924 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:26.924 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:26.924 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:26.924 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:26.924 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:26.924 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:26.924 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:26.924 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:26.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.915+0000 7f6bd5d95700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:26.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.915+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94109176651776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5597800b0400 con 0x55977ffed000 2024-09-13T23:43:26.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.915+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94109176651776] v33'59 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.915+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94109174979328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff82800 con 0x55977ffed400 2024-09-13T23:43:26.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.919+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94109174979328] v33'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.919+0000 7f6bf060ef40 20 remove_watcher() i=0 2024-09-13T23:43:26.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.919+0000 7f6bf060ef40 2 removed watcher, disabling cache 2024-09-13T23:43:26.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.919+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94109174980480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ff83000 con 0x55977ffed000 2024-09-13T23:43:26.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.919+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94109174980480] v33'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ffed000 2024-09-13T23:43:26.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.919+0000 7f6bf060ef40 20 remove_watcher() i=1 2024-09-13T23:43:26.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.919+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94109174981632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55977ffed800 con 0x55977ffed400 2024-09-13T23:43:26.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.923+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94109174981632] v33'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.923+0000 7f6bf060ef40 20 remove_watcher() i=2 2024-09-13T23:43:26.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.923+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94109174982784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55978013e400 con 0x55977ff82400 2024-09-13T23:43:26.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.927+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94109174982784] v33'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.927+0000 7f6bf060ef40 20 remove_watcher() i=3 2024-09-13T23:43:26.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.927+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94109174983936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55978013e800 con 0x55977ffed400 2024-09-13T23:43:26.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.927+0000 7f6beedc7700 1 -- 172.21.15.167:0/3738950467 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94109174983936] v33'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091200 con 0x55977ffed400 2024-09-13T23:43:26.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.927+0000 7f6bf060ef40 20 remove_watcher() i=4 2024-09-13T23:43:26.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.927+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94109174985088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55978013ec00 con 0x55977ff82000 2024-09-13T23:43:26.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.931+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94109174985088] v33'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.931+0000 7f6bf060ef40 20 remove_watcher() i=5 2024-09-13T23:43:26.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.931+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94109174986240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55978013f000 con 0x55977ff82000 2024-09-13T23:43:26.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.931+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94109174986240] v33'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780091440 con 0x55977ff82000 2024-09-13T23:43:26.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.931+0000 7f6bf060ef40 20 remove_watcher() i=6 2024-09-13T23:43:26.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.931+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94109174987392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55978013f400 con 0x55977ff82400 2024-09-13T23:43:26.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bef5c8700 1 -- 172.21.15.167:0/3738950467 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94109174987392] v33'38 uv36 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559780090b40 con 0x55977ff82400 2024-09-13T23:43:26.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 20 remove_watcher() i=7 2024-09-13T23:43:26.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55977ffed400 msgr2=0x55977ff28100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55977ffed400 0x55977ff28100 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55977ffed000 msgr2=0x55977ff27b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55977ffed000 0x55977ff27b80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55977ff82400 msgr2=0x55977ff28c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55977ff82400 0x55977ff28c00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55977ff82000 msgr2=0x55977ff28680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55977ff82000 0x55977ff28680 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55977ffecc00 msgr2=0x55977ff27600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55977ffecc00 0x55977ff27600 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55977ff21920 tx=0x55977fd45800 comp rx=0 tx=0).stop 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 msgr2=0x55977ff26b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3738950467 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41800 0x55977ff26b00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x559780a0bf80 tx=0x55977ffa0840 comp rx=0 tx=0).stop 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 reap_dead start 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bee5c6700 1 -- 172.21.15.167:0/3738950467 reap_dead start 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 shutdown_connections 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 >> 172.21.15.167:0/3738950467 conn(0x55977fc41400 msgr2=0x55977fd41800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 shutdown_connections 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3738950467 wait complete. 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3098043923 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55978013e000 msgr2=0x559780144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3098043923 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55978013e000 0x559780144000 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x55978003bcb0 tx=0x55977ffa17a0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3098043923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 msgr2=0x55977ff29180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3098043923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 0x55977ff29180 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5597800be1b0 tx=0x5597800c0300 comp rx=0 tx=0).stop 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3098043923 shutdown_connections 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3098043923 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55978013e000 0x559780144000 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/3098043923 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5597800b0c00 0x55977ff29180 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3098043923 >> 172.21.15.167:0/3098043923 conn(0x5597800b1c00 msgr2=0x55977fd42000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3098043923 shutdown_connections 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/3098043923 wait complete. 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/4281753315 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55977ffec000 msgr2=0x55977ff27080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/4281753315 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55977ffec000 0x55977ff27080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55977ff6d4d0 tx=0x55977ffa0240 comp rx=0 tx=0).stop 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/4281753315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 msgr2=0x55977ff26580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/4281753315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 0x55977ff26580 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x559780a0be60 tx=0x55977fd45a40 comp rx=0 tx=0).stop 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/4281753315 shutdown_connections 2024-09-13T23:43:26.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/4281753315 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55977ffec000 0x55977ff27080 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 --2- 172.21.15.167:0/4281753315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55977fc41000 0x55977ff26580 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/4281753315 >> 172.21.15.167:0/4281753315 conn(0x55977fc40c00 msgr2=0x55977fd41000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/4281753315 shutdown_connections 2024-09-13T23:43:26.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.935+0000 7f6bf060ef40 1 -- 172.21.15.167:0/4281753315 wait complete. 2024-09-13T23:43:26.950 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:26.950 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-09-13T23:43:26.950 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-09-13T23:43:26.951 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-09-13T23:43:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:26.991 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:26.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e60e5cf40 1 Processor -- start 2024-09-13T23:43:26.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e60e5cf40 1 -- start start 2024-09-13T23:43:26.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e60e5cf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ef11167000 0x55ef1144c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e60e5cf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ef1122b500 con 0x55ef11167400 2024-09-13T23:43:26.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e60e5cf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ef111f4a00 con 0x55ef11167000 2024-09-13T23:43:26.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e5f615700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ef11167000 0x55ef1144c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e5ee14700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55ef11167400 0x55ef11266800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45242/0 (socket says 172.21.15.167:45242) 2024-09-13T23:43:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1199323493 learned_addr learned my addr 172.21.15.167:0/1199323493 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:26.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e5f615700 1 -- 172.21.15.167:0/1199323493 >> v1:172.21.15.12:6789/0 conn(0x55ef11167400 legacy=0x55ef11266800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e5f615700 1 -- 172.21.15.167:0/1199323493 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef11179d40 con 0x55ef11167000 2024-09-13T23:43:26.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e5f615700 1 --2- 172.21.15.167:0/1199323493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167000 0x55ef1144c580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55ef11447a70 tx=0x55ef1126b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d5fa0166cbcc8a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e5e613700 1 -- 172.21.15.167:0/1199323493 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef1120b4a0 con 0x55ef11167000 2024-09-13T23:43:26.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e5e613700 1 -- 172.21.15.167:0/1199323493 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ef1122b800 con 0x55ef11167000 2024-09-13T23:43:26.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e5e613700 1 -- 172.21.15.167:0/1199323493 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef1120b600 con 0x55ef11167000 2024-09-13T23:43:26.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1199323493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167000 msgr2=0x55ef1144c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.983+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1199323493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167000 0x55ef1144c580 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55ef11447a70 tx=0x55ef1126b920 comp rx=0 tx=0).stop 2024-09-13T23:43:26.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1199323493 shutdown_connections 2024-09-13T23:43:26.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1199323493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167000 0x55ef1144c580 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1199323493 >> 172.21.15.167:0/1199323493 conn(0x55ef11166c00 msgr2=0x55ef11266000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1199323493 shutdown_connections 2024-09-13T23:43:26.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1199323493 wait complete. 2024-09-13T23:43:26.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 Processor -- start 2024-09-13T23:43:26.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 -- start start 2024-09-13T23:43:26.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167000 0x55ef1144c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:26.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef111f4500 con 0x55ef11167000 2024-09-13T23:43:26.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e5f615700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167000 0x55ef1144c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:26.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e5f615700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167000 0x55ef1144c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50708/0 (socket says 172.21.15.167:50708) 2024-09-13T23:43:26.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e5f615700 1 -- 172.21.15.167:0/1119845058 learned_addr learned my addr 172.21.15.167:0/1119845058 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e5f615700 1 -- 172.21.15.167:0/1119845058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef11179ba0 con 0x55ef11167000 2024-09-13T23:43:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e5f615700 1 --2- 172.21.15.167:0/1119845058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167000 0x55ef1144c580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55ef11fcb800 tx=0x55ef1126b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bdbda3aee4310e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e5de12700 1 -- 172.21.15.167:0/1119845058 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef1120b760 con 0x55ef11167000 2024-09-13T23:43:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e5de12700 1 -- 172.21.15.167:0/1119845058 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ef1122bb00 con 0x55ef11167000 2024-09-13T23:43:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e5de12700 1 -- 172.21.15.167:0/1119845058 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef1120b4a0 con 0x55ef11167000 2024-09-13T23:43:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1119845058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167000 msgr2=0x55ef1144c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1119845058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167000 0x55ef1144c580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55ef11fcb800 tx=0x55ef1126b800 comp rx=0 tx=0).stop 2024-09-13T23:43:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1119845058 shutdown_connections 2024-09-13T23:43:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1119845058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167000 0x55ef1144c580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1119845058 >> 172.21.15.167:0/1119845058 conn(0x55ef11166c00 msgr2=0x55ef11267000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:26.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1119845058 shutdown_connections 2024-09-13T23:43:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1119845058 wait complete. 2024-09-13T23:43:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.987+0000 7f2e60e5cf40 1 Processor -- start 2024-09-13T23:43:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e60e5cf40 1 -- start start 2024-09-13T23:43:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e60e5cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167400 0x55ef1144cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e60e5cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef111f43c0 con 0x55ef11167400 2024-09-13T23:43:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e5f615700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167400 0x55ef1144cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e5f615700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167400 0x55ef1144cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50710/0 (socket says 172.21.15.167:50710) 2024-09-13T23:43:27.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e5f615700 1 -- 172.21.15.167:0/697875790 learned_addr learned my addr 172.21.15.167:0/697875790 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e5f615700 1 -- 172.21.15.167:0/697875790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef11179a00 con 0x55ef11167400 2024-09-13T23:43:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e5f615700 1 --2- 172.21.15.167:0/697875790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167400 0x55ef1144cb00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55ef11fcb9b0 tx=0x55ef1126b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e5c60f700 1 -- 172.21.15.167:0/697875790 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef1120b600 con 0x55ef11167400 2024-09-13T23:43:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e5c60f700 1 -- 172.21.15.167:0/697875790 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ef1122bc80 con 0x55ef11167400 2024-09-13T23:43:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/697875790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ef11179860 con 0x55ef11167400 2024-09-13T23:43:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e5c60f700 1 -- 172.21.15.167:0/697875790 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef1120b8c0 con 0x55ef11167400 2024-09-13T23:43:27.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/697875790 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef111796c0 con 0x55ef11167400 2024-09-13T23:43:27.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.991+0000 7f2e60e5cf40 1 Processor -- start 2024-09-13T23:43:27.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e60e5cf40 1 -- start start 2024-09-13T23:43:27.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e5c60f700 1 -- 172.21.15.167:0/697875790 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ef11260b00 con 0x55ef11167400 2024-09-13T23:43:27.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e5c60f700 1 --2- 172.21.15.167:0/697875790 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef11167c00 0x55ef1144d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e60e5cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 0x55ef1144d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e5c60f700 1 -- 172.21.15.167:0/697875790 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ef111cd180 con 0x55ef11167400 2024-09-13T23:43:27.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e60e5cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1118c640 con 0x55ef11504000 2024-09-13T23:43:27.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e5fe16700 1 --2- 172.21.15.167:0/697875790 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef11167c00 0x55ef1144d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e5f615700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 0x55ef1144d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e5f615700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 0x55ef1144d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50716/0 (socket says 172.21.15.167:50716) 2024-09-13T23:43:27.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e5f615700 1 -- 172.21.15.167:0/857829588 learned_addr learned my addr 172.21.15.167:0/857829588 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e5f615700 1 -- 172.21.15.167:0/857829588 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef11178000 con 0x55ef11504000 2024-09-13T23:43:27.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e5fe16700 1 --2- 172.21.15.167:0/697875790 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef11167c00 0x55ef1144d080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55ef1148aa80 tx=0x55ef114c0b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e5f615700 1 --2- 172.21.15.167:0/857829588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 0x55ef1144d600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55ef1148af90 tx=0x55ef114c0c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349682fb38894871 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e59e0a700 1 -- 172.21.15.167:0/857829588 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef1120b600 con 0x55ef11504000 2024-09-13T23:43:27.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e59e0a700 1 -- 172.21.15.167:0/857829588 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ef1122bc80 con 0x55ef11504000 2024-09-13T23:43:27.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e59e0a700 1 -- 172.21.15.167:0/857829588 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef1120b8c0 con 0x55ef11504000 2024-09-13T23:43:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/857829588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 msgr2=0x55ef1144d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/857829588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 0x55ef1144d600 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55ef1148af90 tx=0x55ef114c0c60 comp rx=0 tx=0).stop 2024-09-13T23:43:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/857829588 shutdown_connections 2024-09-13T23:43:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/857829588 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 0x55ef1144d600 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/857829588 >> 172.21.15.167:0/857829588 conn(0x55ef11167800 msgr2=0x55ef11267800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/857829588 shutdown_connections 2024-09-13T23:43:27.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.995+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/857829588 wait complete. 2024-09-13T23:43:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e60e5cf40 1 Processor -- start 2024-09-13T23:43:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e60e5cf40 1 -- start start 2024-09-13T23:43:27.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e60e5cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 0x55ef1144d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e60e5cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef1118c8c0 con 0x55ef11504000 2024-09-13T23:43:27.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e5f615700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 0x55ef1144d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e5f615700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 0x55ef1144d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50726/0 (socket says 172.21.15.167:50726) 2024-09-13T23:43:27.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 learned_addr learned my addr 172.21.15.167:0/1970346465 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef11178ea0 con 0x55ef11504000 2024-09-13T23:43:27.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e5f615700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 0x55ef1144d600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55ef1148a7e0 tx=0x55ef114c0e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e58607700 1 -- 172.21.15.167:0/1970346465 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef1120b760 con 0x55ef11504000 2024-09-13T23:43:27.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e58607700 1 -- 172.21.15.167:0/1970346465 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ef1152a000 con 0x55ef11504000 2024-09-13T23:43:27.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ef11178820 con 0x55ef11504000 2024-09-13T23:43:27.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e58607700 1 -- 172.21.15.167:0/1970346465 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef1120b4a0 con 0x55ef11504000 2024-09-13T23:43:27.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:26.999+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef111789c0 con 0x55ef11504000 2024-09-13T23:43:27.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e58607700 1 -- 172.21.15.167:0/1970346465 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ef11260dc0 con 0x55ef11504000 2024-09-13T23:43:27.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e58607700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef11505800 0x55ef1144e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e58607700 1 -- 172.21.15.167:0/1970346465 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ef111cd340 con 0x55ef11504000 2024-09-13T23:43:27.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ef11504400 0x55ef1144db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e5ee14700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef11505800 0x55ef1144e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef11504800 con 0x55ef11504400 2024-09-13T23:43:27.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e5fe16700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ef11504400 0x55ef1144db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e5ee14700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef11505800 0x55ef1144e680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55ef114e7d10 tx=0x55ef1159a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e5fe16700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ef11504400 0x55ef1144db80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 20 realm 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef11504c00 con 0x55ef11504400 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef11504800 con 0x55ef11504400 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ef11504c00 0x55ef1144e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef11505c00 con 0x55ef11504c00 2024-09-13T23:43:27.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e5f615700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ef11504c00 0x55ef1144e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.003+0000 7f2e5f615700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ef11504c00 0x55ef1144e100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e60e5cf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:27.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e60e5cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6000 con 0x55ef11504c00 2024-09-13T23:43:27.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e60e5cf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:27.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e60e5cf40 20 searching for the correct realm 2024-09-13T23:43:27.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ef115a6400 0x55ef1144ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6400 2024-09-13T23:43:27.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e5ee14700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ef115a6400 0x55ef1144ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e5ee14700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ef115a6400 0x55ef1144ec00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504c00 2024-09-13T23:43:27.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e5f615700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ef115a6000 0x55ef1144c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6000 2024-09-13T23:43:27.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.007+0000 7f2e5fe16700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ef115a6000 0x55ef1144c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5fe16700 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ef115a6000 0x55ef1144c580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504400 2024-09-13T23:43:27.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504400 2024-09-13T23:43:27.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6400 2024-09-13T23:43:27.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504400 2024-09-13T23:43:27.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504c00 2024-09-13T23:43:27.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504c00 2024-09-13T23:43:27.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.011+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504400 2024-09-13T23:43:27.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6000 2024-09-13T23:43:27.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504400 2024-09-13T23:43:27.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6000 2024-09-13T23:43:27.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504c00 2024-09-13T23:43:27.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6400 2024-09-13T23:43:27.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504400 2024-09-13T23:43:27.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.015+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504c00 2024-09-13T23:43:27.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.019+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.019+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6400 2024-09-13T23:43:27.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.019+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.019+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504400 2024-09-13T23:43:27.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.019+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.019+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504400 2024-09-13T23:43:27.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.019+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.019+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504400 2024-09-13T23:43:27.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.019+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.019+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6400 2024-09-13T23:43:27.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.019+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.019+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6000 2024-09-13T23:43:27.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504400 2024-09-13T23:43:27.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6000 2024-09-13T23:43:27.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6400 2024-09-13T23:43:27.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6000 2024-09-13T23:43:27.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504c00 2024-09-13T23:43:27.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504c00 2024-09-13T23:43:27.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.023+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef115a6400 2024-09-13T23:43:27.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504c00 2024-09-13T23:43:27.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504c00 2024-09-13T23:43:27.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e60e5cf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:27.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e60e5cf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:27.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e60e5cf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:27.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e60e5cf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:27.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e60e5cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6c00 con 0x55ef11504400 2024-09-13T23:43:27.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e60e5cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e60e5cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7000 con 0x55ef11504c00 2024-09-13T23:43:27.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e60e5cf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:27.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e60e5cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.027+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7400 con 0x55ef11504c00 2024-09-13T23:43:27.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:27.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 20 zone default found 2024-09-13T23:43:27.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 4 Realm: () 2024-09-13T23:43:27.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:27.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:27.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:27.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 20 zonegroup default 2024-09-13T23:43:27.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7800 con 0x55ef115a6000 2024-09-13T23:43:27.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7c00 con 0x55ef115a6400 2024-09-13T23:43:27.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6c00 con 0x55ef115a6400 2024-09-13T23:43:27.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 20 started sync module instance, tier type = 2024-09-13T23:43:27.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:27.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.031+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7800 con 0x55ef11504c00 2024-09-13T23:43:27.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.035+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.035+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94485275228928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef11504800 con 0x55ef11504c00 2024-09-13T23:43:27.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.035+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504400 2024-09-13T23:43:27.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.035+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.035+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94485275230080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7000 con 0x55ef11504400 2024-09-13T23:43:27.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.035+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7400 con 0x55ef11504c00 2024-09-13T23:43:27.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.035+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94485275228928] v33'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.039+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94485275230080] v33'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.039+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.039+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94485275231232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7800 con 0x55ef11504c00 2024-09-13T23:43:27.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.039+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7c00 con 0x55ef115a6000 2024-09-13T23:43:27.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.043+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.043+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94485275232384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6c00 con 0x55ef115a6000 2024-09-13T23:43:27.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.043+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef11505c00 con 0x55ef11504c00 2024-09-13T23:43:27.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.043+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94485275231232] v33'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.043+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.043+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94485275232384] v33'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.043+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94485275233536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6800 con 0x55ef11504c00 2024-09-13T23:43:27.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.043+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7000 con 0x55ef115a6400 2024-09-13T23:43:27.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.047+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.047+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94485275234688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7c00 con 0x55ef115a6400 2024-09-13T23:43:27.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.047+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6c00 con 0x55ef115a6400 2024-09-13T23:43:27.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.047+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94485275233536] v33'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.047+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.047+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94485275235840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7000 con 0x55ef115a6400 2024-09-13T23:43:27.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.047+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7c00 con 0x55ef115a6000 2024-09-13T23:43:27.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.051+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94485275234688] v33'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.051+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'39 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.051+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94485275235840] v33'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.051+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94485275236992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6c00 con 0x55ef115a6000 2024-09-13T23:43:27.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.051+0000 7f2e60e5cf40 20 add_watcher() i=0 2024-09-13T23:43:27.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.051+0000 7f2e60e5cf40 20 add_watcher() i=1 2024-09-13T23:43:27.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.051+0000 7f2e60e5cf40 20 add_watcher() i=2 2024-09-13T23:43:27.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.051+0000 7f2e60e5cf40 20 add_watcher() i=3 2024-09-13T23:43:27.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.051+0000 7f2e60e5cf40 20 add_watcher() i=4 2024-09-13T23:43:27.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.051+0000 7f2e60e5cf40 20 add_watcher() i=5 2024-09-13T23:43:27.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.051+0000 7f2e60e5cf40 20 add_watcher() i=6 2024-09-13T23:43:27.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.055+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94485275236992] v33'40 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.055+0000 7f2e60e5cf40 20 add_watcher() i=7 2024-09-13T23:43:27.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.055+0000 7f2e60e5cf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:27.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.055+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115b6000 con 0x55ef11504400 2024-09-13T23:43:27.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.055+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.055+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94485276762112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115b6400 con 0x55ef11504400 2024-09-13T23:43:27.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.055+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94485276762112] v33'60 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.055+0000 7f2e465e3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:27.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.055+0000 7f2e60e5cf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:27.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.055+0000 7f2e60e5cf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:27.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e60e5cf40 1 Processor -- start 2024-09-13T23:43:27.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e60e5cf40 1 -- start start 2024-09-13T23:43:27.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e60e5cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 0x55ef1144f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e60e5cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef111f43c0 con 0x55ef115b6c00 2024-09-13T23:43:27.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e5ee14700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 0x55ef1144f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e5ee14700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 0x55ef1144f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50738/0 (socket says 172.21.15.167:50738) 2024-09-13T23:43:27.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e5ee14700 1 -- 172.21.15.167:0/403920045 learned_addr learned my addr 172.21.15.167:0/403920045 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e5ee14700 1 -- 172.21.15.167:0/403920045 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef11178b60 con 0x55ef115b6c00 2024-09-13T23:43:27.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e5ee14700 1 --2- 172.21.15.167:0/403920045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 0x55ef1144f180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55ef115c2090 tx=0x55ef114c1f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbaa706516cfa80 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e45de2700 1 -- 172.21.15.167:0/403920045 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef115b8160 con 0x55ef115b6c00 2024-09-13T23:43:27.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e45de2700 1 -- 172.21.15.167:0/403920045 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ef1152aa80 con 0x55ef115b6c00 2024-09-13T23:43:27.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e45de2700 1 -- 172.21.15.167:0/403920045 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef115b82c0 con 0x55ef115b6c00 2024-09-13T23:43:27.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/403920045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 msgr2=0x55ef1144f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/403920045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 0x55ef1144f180 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55ef115c2090 tx=0x55ef114c1f20 comp rx=0 tx=0).stop 2024-09-13T23:43:27.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/403920045 shutdown_connections 2024-09-13T23:43:27.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/403920045 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 0x55ef1144f180 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/403920045 >> 172.21.15.167:0/403920045 conn(0x55ef115b6800 msgr2=0x55ef11268000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/403920045 shutdown_connections 2024-09-13T23:43:27.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.059+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/403920045 wait complete. 2024-09-13T23:43:27.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e60e5cf40 1 Processor -- start 2024-09-13T23:43:27.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e60e5cf40 1 -- start start 2024-09-13T23:43:27.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e60e5cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 0x55ef1144f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e60e5cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef111f4500 con 0x55ef115b6c00 2024-09-13T23:43:27.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e5ee14700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 0x55ef1144f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e5ee14700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 0x55ef1144f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50742/0 (socket says 172.21.15.167:50742) 2024-09-13T23:43:27.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1023150026 learned_addr learned my addr 172.21.15.167:0/1023150026 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1023150026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef11178d00 con 0x55ef115b6c00 2024-09-13T23:43:27.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e5ee14700 1 --2- 172.21.15.167:0/1023150026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 0x55ef1144f180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55ef115c3350 tx=0x55ef1159a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e445df700 1 -- 172.21.15.167:0/1023150026 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef115b8420 con 0x55ef115b6c00 2024-09-13T23:43:27.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e445df700 1 -- 172.21.15.167:0/1023150026 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ef1152a780 con 0x55ef115b6c00 2024-09-13T23:43:27.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e445df700 1 -- 172.21.15.167:0/1023150026 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ef115b8160 con 0x55ef115b6c00 2024-09-13T23:43:27.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1023150026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ef11178340 con 0x55ef115b6c00 2024-09-13T23:43:27.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.063+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1023150026 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef11178b60 con 0x55ef115b6c00 2024-09-13T23:43:27.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.067+0000 7f2e445df700 1 -- 172.21.15.167:0/1023150026 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ef11261b80 con 0x55ef115b6c00 2024-09-13T23:43:27.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.067+0000 7f2e445df700 1 --2- 172.21.15.167:0/1023150026 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef1163e000 0x55ef11644000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.067+0000 7f2e445df700 1 -- 172.21.15.167:0/1023150026 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ef111cd6c0 con 0x55ef115b6c00 2024-09-13T23:43:27.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.067+0000 7f2e5fe16700 1 --2- 172.21.15.167:0/1023150026 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef1163e000 0x55ef11644000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.067+0000 7f2e60e5cf40 5 note: GC not initialized 2024-09-13T23:43:27.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.067+0000 7f2e5fe16700 1 --2- 172.21.15.167:0/1023150026 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef1163e000 0x55ef11644000 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55ef11549e60 tx=0x55ef114c0a80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.067+0000 7f2e41dda700 20 reqs_thread_entry: start 2024-09-13T23:43:27.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.067+0000 7f2e60e5cf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:27.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.067+0000 7f2e60e5cf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:27.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.067+0000 7f2e3f5d5700 20 reqs_thread_entry: start 2024-09-13T23:43:27.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.067+0000 7f2e3fdd6700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:57 2024 2024-09-13T23:43:27.078 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:27.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.067+0000 7f2e60e5cf40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:27.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.071+0000 7f2e60e5cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.071+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115b7800 con 0x55ef11504400 2024-09-13T23:43:27.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.071+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.071+0000 7f2e60e5cf40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:27.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.071+0000 7f2e60e5cf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:27.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.071+0000 7f2e60e5cf40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:27.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.071+0000 7f2e60e5cf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:27.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.071+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115b7c00 con 0x55ef115a6000 2024-09-13T23:43:27.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.071+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.071+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef1163e800 con 0x55ef11504400 2024-09-13T23:43:27.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~357] v33'3 uv3 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e60e5cf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:27.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e60e5cf40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:27.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e60e5cf40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:27.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94485276764416 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef1163ec00 con 0x55ef11504c00 2024-09-13T23:43:27.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94485275233536 notify 141733920770 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x55ef111796c0 con 0x55ef11504c00 2024-09-13T23:43:27.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e59609700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920770 cookie 94485275233536 notifier 4334 bl.length()=533 2024-09-13T23:43:27.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 notify.4 [notify cookie 94485276764416 out=8b] v0'0 uv39 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e59609700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:27.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e59609700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:27.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e59609700 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef11504800 con 0x55ef11504c00 2024-09-13T23:43:27.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== watch-notify(notify_complete (2) cookie 94485276764416 notify 141733920770 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ef11179d40 con 0x55ef11504c00 2024-09-13T23:43:27.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.075+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.085 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:27.085 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:27.086 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:27.087 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:27.088 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:27.088 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:27.088 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:27.088 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:27.088 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:27.088 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:27.088 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:27.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.079+0000 7f2e465e3700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:27.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.079+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94485276762112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef1163f000 con 0x55ef11504400 2024-09-13T23:43:27.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.079+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94485276762112] v33'61 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.079+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94485275228928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef1163f400 con 0x55ef11504c00 2024-09-13T23:43:27.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.083+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94485275228928] v33'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.083+0000 7f2e60e5cf40 20 remove_watcher() i=0 2024-09-13T23:43:27.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.083+0000 7f2e60e5cf40 2 removed watcher, disabling cache 2024-09-13T23:43:27.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.083+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94485275230080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef1163f800 con 0x55ef11504400 2024-09-13T23:43:27.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.083+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94485275230080] v33'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef11504400 2024-09-13T23:43:27.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.083+0000 7f2e60e5cf40 20 remove_watcher() i=1 2024-09-13T23:43:27.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.083+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94485275231232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef1163fc00 con 0x55ef11504c00 2024-09-13T23:43:27.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.087+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94485275231232] v33'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.087+0000 7f2e60e5cf40 20 remove_watcher() i=2 2024-09-13T23:43:27.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.087+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94485275232384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115b6000 con 0x55ef115a6000 2024-09-13T23:43:27.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.087+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94485275232384] v33'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.087+0000 7f2e60e5cf40 20 remove_watcher() i=3 2024-09-13T23:43:27.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.087+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94485275233536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115b6400 con 0x55ef11504c00 2024-09-13T23:43:27.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.091+0000 7f2e5f615700 1 -- 172.21.15.167:0/1970346465 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94485275233536] v33'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597200 con 0x55ef11504c00 2024-09-13T23:43:27.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.091+0000 7f2e60e5cf40 20 remove_watcher() i=4 2024-09-13T23:43:27.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.091+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94485275234688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115b7800 con 0x55ef115a6400 2024-09-13T23:43:27.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.091+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94485275234688] v33'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.091+0000 7f2e60e5cf40 20 remove_watcher() i=5 2024-09-13T23:43:27.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.091+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94485275235840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a7c00 con 0x55ef115a6400 2024-09-13T23:43:27.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.095+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94485275235840] v33'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11597440 con 0x55ef115a6400 2024-09-13T23:43:27.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.095+0000 7f2e60e5cf40 20 remove_watcher() i=6 2024-09-13T23:43:27.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.095+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94485275236992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ef115a6c00 con 0x55ef115a6000 2024-09-13T23:43:27.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e5fe16700 1 -- 172.21.15.167:0/1970346465 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94485275236992] v33'41 uv39 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ef11596b40 con 0x55ef115a6000 2024-09-13T23:43:27.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 20 remove_watcher() i=7 2024-09-13T23:43:27.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ef11504c00 msgr2=0x55ef1144e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ef11504c00 0x55ef1144e100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ef11504400 msgr2=0x55ef1144db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ef11504400 0x55ef1144db80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ef115a6000 msgr2=0x55ef1144c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ef115a6000 0x55ef1144c580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ef115a6400 msgr2=0x55ef1144ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ef115a6400 0x55ef1144ec00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef11505800 msgr2=0x55ef1144e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef11505800 0x55ef1144e680 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55ef114e7d10 tx=0x55ef1159a000 comp rx=0 tx=0).stop 2024-09-13T23:43:27.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 msgr2=0x55ef1144d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e5ee14700 1 -- 172.21.15.167:0/1970346465 reap_dead start 2024-09-13T23:43:27.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 0x55ef1144d600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55ef1148a7e0 tx=0x55ef114c0e40 comp rx=0 tx=0).stop 2024-09-13T23:43:27.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 shutdown_connections 2024-09-13T23:43:27.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1970346465 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11504000 0x55ef1144d600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 >> 172.21.15.167:0/1970346465 conn(0x55ef11167800 msgr2=0x55ef11267800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 shutdown_connections 2024-09-13T23:43:27.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1970346465 wait complete. 2024-09-13T23:43:27.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1023150026 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef1163e000 msgr2=0x55ef11644000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1023150026 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef1163e000 0x55ef11644000 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55ef11549e60 tx=0x55ef114c0a80 comp rx=0 tx=0).stop 2024-09-13T23:43:27.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1023150026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 msgr2=0x55ef1144f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1023150026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 0x55ef1144f180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55ef115c3350 tx=0x55ef1159a0c0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1023150026 shutdown_connections 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1023150026 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef1163e000 0x55ef11644000 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/1023150026 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef115b6c00 0x55ef1144f180 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1023150026 >> 172.21.15.167:0/1023150026 conn(0x55ef115b6800 msgr2=0x55ef11268000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1023150026 shutdown_connections 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/1023150026 wait complete. 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/697875790 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef11167c00 msgr2=0x55ef1144d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/697875790 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef11167c00 0x55ef1144d080 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55ef1148aa80 tx=0x55ef114c0b40 comp rx=0 tx=0).stop 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/697875790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167400 msgr2=0x55ef1144cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/697875790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167400 0x55ef1144cb00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55ef11fcb9b0 tx=0x55ef1126b920 comp rx=0 tx=0).stop 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/697875790 shutdown_connections 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/697875790 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ef11167c00 0x55ef1144d080 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 --2- 172.21.15.167:0/697875790 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ef11167400 0x55ef1144cb00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/697875790 >> 172.21.15.167:0/697875790 conn(0x55ef11166c00 msgr2=0x55ef11267000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/697875790 shutdown_connections 2024-09-13T23:43:27.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.099+0000 7f2e60e5cf40 1 -- 172.21.15.167:0/697875790 wait complete. 2024-09-13T23:43:27.115 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:27.115 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-13T23:43:27.115 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-13T23:43:27.115 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-13T23:43:27.200 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:27.201 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:27.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.191+0000 7f11b2f40f40 1 Processor -- start 2024-09-13T23:43:27.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.191+0000 7f11b2f40f40 1 -- start start 2024-09-13T23:43:27.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.191+0000 7f11b2f40f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55a507e3b400 0x55a508120580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.191+0000 7f11b2f40f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a507eff500 con 0x55a507e3b000 2024-09-13T23:43:27.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.191+0000 7f11b2f40f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a507ec8a00 con 0x55a507e3b400 2024-09-13T23:43:27.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.191+0000 7f11b0ef8700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55a507e3b400 0x55a508120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.191+0000 7f11b16f9700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55a507e3b000 0x55a507f3a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45250/0 (socket says 172.21.15.167:45250) 2024-09-13T23:43:27.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.191+0000 7f11b16f9700 1 -- 172.21.15.167:0/863734027 learned_addr learned my addr 172.21.15.167:0/863734027 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:27.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b0ef8700 1 -- 172.21.15.167:0/863734027 >> v1:172.21.15.12:6789/0 conn(0x55a507e3b000 legacy=0x55a507f3a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b0ef8700 1 -- 172.21.15.167:0/863734027 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a507e4dd40 con 0x55a507e3b400 2024-09-13T23:43:27.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b0ef8700 1 --2- 172.21.15.167:0/863734027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b400 0x55a508120580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55a50811b920 tx=0x55a507f3f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fd115e7d0cc322 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b06f7700 1 -- 172.21.15.167:0/863734027 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a507edf4a0 con 0x55a507e3b400 2024-09-13T23:43:27.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b06f7700 1 -- 172.21.15.167:0/863734027 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a507eff800 con 0x55a507e3b400 2024-09-13T23:43:27.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b06f7700 1 -- 172.21.15.167:0/863734027 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a507edf600 con 0x55a507e3b400 2024-09-13T23:43:27.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b2f40f40 1 -- 172.21.15.167:0/863734027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b400 msgr2=0x55a508120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/863734027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b400 0x55a508120580 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55a50811b920 tx=0x55a507f3f920 comp rx=0 tx=0).stop 2024-09-13T23:43:27.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b2f40f40 1 -- 172.21.15.167:0/863734027 shutdown_connections 2024-09-13T23:43:27.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/863734027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b400 0x55a508120580 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b2f40f40 1 -- 172.21.15.167:0/863734027 >> 172.21.15.167:0/863734027 conn(0x55a507e3ac00 msgr2=0x55a507f3a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b2f40f40 1 -- 172.21.15.167:0/863734027 shutdown_connections 2024-09-13T23:43:27.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b2f40f40 1 -- 172.21.15.167:0/863734027 wait complete. 2024-09-13T23:43:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b2f40f40 1 Processor -- start 2024-09-13T23:43:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b2f40f40 1 -- start start 2024-09-13T23:43:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b2f40f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 0x55a508120580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b2f40f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a507ec8500 con 0x55a507e3b000 2024-09-13T23:43:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b16f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 0x55a508120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b16f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 0x55a508120580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50768/0 (socket says 172.21.15.167:50768) 2024-09-13T23:43:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b16f9700 1 -- 172.21.15.167:0/2161535083 learned_addr learned my addr 172.21.15.167:0/2161535083 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.195+0000 7f11b16f9700 1 -- 172.21.15.167:0/2161535083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a507e4dba0 con 0x55a507e3b000 2024-09-13T23:43:27.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b16f9700 1 --2- 172.21.15.167:0/2161535083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 0x55a508120580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55a508c9f8c0 tx=0x55a507f3fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2edb3d55bc35d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11afef6700 1 -- 172.21.15.167:0/2161535083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a507edf4a0 con 0x55a507e3b000 2024-09-13T23:43:27.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11afef6700 1 -- 172.21.15.167:0/2161535083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a507effb00 con 0x55a507e3b000 2024-09-13T23:43:27.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11afef6700 1 -- 172.21.15.167:0/2161535083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a507edf600 con 0x55a507e3b000 2024-09-13T23:43:27.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2161535083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 msgr2=0x55a508120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2161535083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 0x55a508120580 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55a508c9f8c0 tx=0x55a507f3fb60 comp rx=0 tx=0).stop 2024-09-13T23:43:27.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2161535083 shutdown_connections 2024-09-13T23:43:27.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2161535083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 0x55a508120580 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2161535083 >> 172.21.15.167:0/2161535083 conn(0x55a507e3ac00 msgr2=0x55a507f3b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2161535083 shutdown_connections 2024-09-13T23:43:27.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2161535083 wait complete. 2024-09-13T23:43:27.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b2f40f40 1 Processor -- start 2024-09-13T23:43:27.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b2f40f40 1 -- start start 2024-09-13T23:43:27.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b2f40f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 0x55a508120580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b2f40f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a507ec83c0 con 0x55a507e3b000 2024-09-13T23:43:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b16f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 0x55a508120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b16f9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 0x55a508120580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50770/0 (socket says 172.21.15.167:50770) 2024-09-13T23:43:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.199+0000 7f11b16f9700 1 -- 172.21.15.167:0/3300350869 learned_addr learned my addr 172.21.15.167:0/3300350869 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b16f9700 1 -- 172.21.15.167:0/3300350869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a507e4da00 con 0x55a507e3b000 2024-09-13T23:43:27.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b16f9700 1 --2- 172.21.15.167:0/3300350869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 0x55a508120580 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55a508c9faa0 tx=0x55a507f3fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11ae6f3700 1 -- 172.21.15.167:0/3300350869 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a507edf8c0 con 0x55a507e3b000 2024-09-13T23:43:27.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11ae6f3700 1 -- 172.21.15.167:0/3300350869 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a507effc80 con 0x55a507e3b000 2024-09-13T23:43:27.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3300350869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a507e4d860 con 0x55a507e3b000 2024-09-13T23:43:27.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11ae6f3700 1 -- 172.21.15.167:0/3300350869 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a507edf4a0 con 0x55a507e3b000 2024-09-13T23:43:27.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3300350869 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a507e4d6c0 con 0x55a507e3b000 2024-09-13T23:43:27.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b2f40f40 1 Processor -- start 2024-09-13T23:43:27.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b2f40f40 1 -- start start 2024-09-13T23:43:27.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b2f40f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 0x55a508120b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b2f40f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a507ec8000 con 0x55a507e3b800 2024-09-13T23:43:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11ae6f3700 1 -- 172.21.15.167:0/3300350869 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55a507f34b00 con 0x55a507e3b000 2024-09-13T23:43:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b1efa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 0x55a508120b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b1efa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 0x55a508120b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50776/0 (socket says 172.21.15.167:50776) 2024-09-13T23:43:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b1efa700 1 -- 172.21.15.167:0/2941366018 learned_addr learned my addr 172.21.15.167:0/2941366018 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11ae6f3700 1 --2- 172.21.15.167:0/3300350869 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a5081ec000 0x55a508121080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11ae6f3700 1 -- 172.21.15.167:0/3300350869 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a507ea1180 con 0x55a507e3b000 2024-09-13T23:43:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b16f9700 1 --2- 172.21.15.167:0/3300350869 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a5081ec000 0x55a508121080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b1efa700 1 -- 172.21.15.167:0/2941366018 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a507e4c000 con 0x55a507e3b800 2024-09-13T23:43:27.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b1efa700 1 --2- 172.21.15.167:0/2941366018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 0x55a508120b00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55a5081c6510 tx=0x55a5081a2de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b971a25ec0f257df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.203+0000 7f11b16f9700 1 --2- 172.21.15.167:0/3300350869 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a5081ec000 0x55a508121080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55a508cb35f0 tx=0x55a5081a2240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11abeee700 1 -- 172.21.15.167:0/2941366018 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a507edf8c0 con 0x55a507e3b800 2024-09-13T23:43:27.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11abeee700 1 -- 172.21.15.167:0/2941366018 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a5081fa000 con 0x55a507e3b800 2024-09-13T23:43:27.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11abeee700 1 -- 172.21.15.167:0/2941366018 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a507edf4a0 con 0x55a507e3b800 2024-09-13T23:43:27.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2941366018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 msgr2=0x55a508120b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2941366018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 0x55a508120b00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55a5081c6510 tx=0x55a5081a2de0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2941366018 shutdown_connections 2024-09-13T23:43:27.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2941366018 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 0x55a508120b00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2941366018 >> 172.21.15.167:0/2941366018 conn(0x55a507e3b400 msgr2=0x55a507f3b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2941366018 shutdown_connections 2024-09-13T23:43:27.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2941366018 wait complete. 2024-09-13T23:43:27.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 Processor -- start 2024-09-13T23:43:27.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 -- start start 2024-09-13T23:43:27.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 0x55a508120b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a507e60640 con 0x55a507e3b800 2024-09-13T23:43:27.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b1efa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 0x55a508120b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b1efa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 0x55a508120b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50778/0 (socket says 172.21.15.167:50778) 2024-09-13T23:43:27.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 learned_addr learned my addr 172.21.15.167:0/2605329865 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a507e4cea0 con 0x55a507e3b800 2024-09-13T23:43:27.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b1efa700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 0x55a508120b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55a508cb2420 tx=0x55a5081a2840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11aa6eb700 1 -- 172.21.15.167:0/2605329865 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a507edfb80 con 0x55a507e3b800 2024-09-13T23:43:27.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a507e4c820 con 0x55a507e3b800 2024-09-13T23:43:27.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11aa6eb700 1 -- 172.21.15.167:0/2605329865 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a5081fa180 con 0x55a507e3b800 2024-09-13T23:43:27.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11aa6eb700 1 -- 172.21.15.167:0/2605329865 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a507edf8c0 con 0x55a507e3b800 2024-09-13T23:43:27.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.207+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a507e4c9c0 con 0x55a507e3b800 2024-09-13T23:43:27.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11aa6eb700 1 -- 172.21.15.167:0/2605329865 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55a507f34dc0 con 0x55a507e3b800 2024-09-13T23:43:27.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11aa6eb700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a5081ecc00 0x55a508121600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11aa6eb700 1 -- 172.21.15.167:0/2605329865 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a507ea1340 con 0x55a507e3b800 2024-09-13T23:43:27.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b0ef8700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a5081ecc00 0x55a508121600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b2f40f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a5081ed000 0x55a508121b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed400 con 0x55a5081ed000 2024-09-13T23:43:27.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b1efa700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a5081ed000 0x55a508121b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b0ef8700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a5081ecc00 0x55a508121600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55a50811b920 tx=0x55a507f3f800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b1efa700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a5081ed000 0x55a508121b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b2f40f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b2f40f40 20 realm 2024-09-13T23:43:27.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b2f40f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed000 2024-09-13T23:43:27.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b2f40f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b2f40f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:27.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b2f40f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.211+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081edc00 con 0x55a5081ed000 2024-09-13T23:43:27.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b2f40f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b2f40f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a5081ed400 0x55a508122100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed400 2024-09-13T23:43:27.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b16f9700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a5081ed400 0x55a508122100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b16f9700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a5081ed400 0x55a508122100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b2f40f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:27.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b2f40f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081edc00 con 0x55a5081ed400 2024-09-13T23:43:27.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b2f40f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:27.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b2f40f40 20 searching for the correct realm 2024-09-13T23:43:27.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a508172000 0x55a508122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508172400 con 0x55a508172000 2024-09-13T23:43:27.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b0ef8700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a508172000 0x55a508122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.215+0000 7f11b0ef8700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a508172000 0x55a508122680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508172400 con 0x55a5081ed400 2024-09-13T23:43:27.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b16f9700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a508172400 0x55a508122c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172400 2024-09-13T23:43:27.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b1efa700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a508172400 0x55a508122c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b1efa700 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a508172400 0x55a508122c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed000 2024-09-13T23:43:27.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed000 2024-09-13T23:43:27.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172000 2024-09-13T23:43:27.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.219+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed000 2024-09-13T23:43:27.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed400 2024-09-13T23:43:27.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed400 2024-09-13T23:43:27.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed000 2024-09-13T23:43:27.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172400 2024-09-13T23:43:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed000 2024-09-13T23:43:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172400 2024-09-13T23:43:27.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.223+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed400 2024-09-13T23:43:27.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172000 2024-09-13T23:43:27.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed000 2024-09-13T23:43:27.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed400 2024-09-13T23:43:27.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172000 2024-09-13T23:43:27.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed000 2024-09-13T23:43:27.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed000 2024-09-13T23:43:27.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.227+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed000 2024-09-13T23:43:27.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172000 2024-09-13T23:43:27.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172400 2024-09-13T23:43:27.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed000 2024-09-13T23:43:27.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172400 2024-09-13T23:43:27.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172000 2024-09-13T23:43:27.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172400 2024-09-13T23:43:27.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.231+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed400 2024-09-13T23:43:27.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed400 2024-09-13T23:43:27.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172000 2024-09-13T23:43:27.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed400 2024-09-13T23:43:27.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed400 2024-09-13T23:43:27.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b2f40f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:27.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b2f40f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:27.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b2f40f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:27.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b2f40f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:27.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b2f40f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508172800 con 0x55a5081ed000 2024-09-13T23:43:27.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b2f40f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b2f40f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.235+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508172c00 con 0x55a5081ed400 2024-09-13T23:43:27.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:27.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508173000 con 0x55a5081ed400 2024-09-13T23:43:27.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:27.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 zone default found 2024-09-13T23:43:27.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 4 Realm: () 2024-09-13T23:43:27.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:27.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:27.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:27.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 zonegroup default 2024-09-13T23:43:27.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508173400 con 0x55a508172400 2024-09-13T23:43:27.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508173800 con 0x55a508172000 2024-09-13T23:43:27.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508173c00 con 0x55a508172000 2024-09-13T23:43:27.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 started sync module instance, tier type = 2024-09-13T23:43:27.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:27.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.239+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508173800 con 0x55a5081ed400 2024-09-13T23:43:27.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.243+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.243+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94167293328128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508173c00 con 0x55a5081ed400 2024-09-13T23:43:27.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.243+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508172c00 con 0x55a5081ed000 2024-09-13T23:43:27.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.247+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.247+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94167293329280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508173000 con 0x55a5081ed000 2024-09-13T23:43:27.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.247+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94167293328128] v33'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.247+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508173800 con 0x55a5081ed400 2024-09-13T23:43:27.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.247+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94167293329280] v33'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.247+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.247+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94167293330432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081edc00 con 0x55a5081ed400 2024-09-13T23:43:27.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.247+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a508172400 2024-09-13T23:43:27.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.251+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.251+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94167293331584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508172800 con 0x55a508172400 2024-09-13T23:43:27.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.251+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508173400 con 0x55a5081ed400 2024-09-13T23:43:27.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.251+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94167293330432] v33'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.251+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94167293331584] v33'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.251+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.251+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94167293332736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508172c00 con 0x55a5081ed400 2024-09-13T23:43:27.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.251+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508173000 con 0x55a508172000 2024-09-13T23:43:27.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.255+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.255+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94167293333888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5082ac000 con 0x55a508172000 2024-09-13T23:43:27.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.255+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5082ac400 con 0x55a508172000 2024-09-13T23:43:27.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.255+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94167293332736] v33'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.255+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.255+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94167293335040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5082ac800 con 0x55a508172000 2024-09-13T23:43:27.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.255+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5082acc00 con 0x55a508172400 2024-09-13T23:43:27.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.259+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94167293333888] v33'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.259+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94167293335040] v33'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.259+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'42 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.259+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94167293336192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5082ad000 con 0x55a508172400 2024-09-13T23:43:27.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.259+0000 7f11b2f40f40 20 add_watcher() i=0 2024-09-13T23:43:27.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.259+0000 7f11b2f40f40 20 add_watcher() i=1 2024-09-13T23:43:27.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.259+0000 7f11b2f40f40 20 add_watcher() i=2 2024-09-13T23:43:27.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.259+0000 7f11b2f40f40 20 add_watcher() i=3 2024-09-13T23:43:27.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.259+0000 7f11b2f40f40 20 add_watcher() i=4 2024-09-13T23:43:27.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.259+0000 7f11b2f40f40 20 add_watcher() i=5 2024-09-13T23:43:27.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.259+0000 7f11b2f40f40 20 add_watcher() i=6 2024-09-13T23:43:27.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.259+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94167293336192] v33'43 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11b2f40f40 20 add_watcher() i=7 2024-09-13T23:43:27.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11b2f40f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:27.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5082ad400 con 0x55a5081ed000 2024-09-13T23:43:27.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94167295008768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5082ad800 con 0x55a5081ed000 2024-09-13T23:43:27.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94167295008768] v33'62 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11986c7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:27.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11b2f40f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:27.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11b2f40f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:27.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11b2f40f40 1 Processor -- start 2024-09-13T23:43:27.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11b2f40f40 1 -- start start 2024-09-13T23:43:27.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11b2f40f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 0x55a508123180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.263+0000 7f11b2f40f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a507e60780 con 0x55a5082acc00 2024-09-13T23:43:27.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b0ef8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 0x55a508123180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b0ef8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 0x55a508123180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50784/0 (socket says 172.21.15.167:50784) 2024-09-13T23:43:27.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b0ef8700 1 -- 172.21.15.167:0/3342465483 learned_addr learned my addr 172.21.15.167:0/3342465483 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b0ef8700 1 -- 172.21.15.167:0/3342465483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a507e4cb60 con 0x55a5082acc00 2024-09-13T23:43:27.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b0ef8700 1 --2- 172.21.15.167:0/3342465483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 0x55a508123180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55a5082ba0f0 tx=0x55a507f3f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5926bd2297fcb584 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f1197ec6700 1 -- 172.21.15.167:0/3342465483 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a507edf760 con 0x55a5082acc00 2024-09-13T23:43:27.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f1197ec6700 1 -- 172.21.15.167:0/3342465483 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a5081fa600 con 0x55a5082acc00 2024-09-13T23:43:27.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f1197ec6700 1 -- 172.21.15.167:0/3342465483 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a5082c6000 con 0x55a5082acc00 2024-09-13T23:43:27.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3342465483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 msgr2=0x55a508123180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/3342465483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 0x55a508123180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55a5082ba0f0 tx=0x55a507f3f860 comp rx=0 tx=0).stop 2024-09-13T23:43:27.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3342465483 shutdown_connections 2024-09-13T23:43:27.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/3342465483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 0x55a508123180 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3342465483 >> 172.21.15.167:0/3342465483 conn(0x55a5082adc00 msgr2=0x55a507f3c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3342465483 shutdown_connections 2024-09-13T23:43:27.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3342465483 wait complete. 2024-09-13T23:43:27.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b2f40f40 1 Processor -- start 2024-09-13T23:43:27.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b2f40f40 1 -- start start 2024-09-13T23:43:27.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b2f40f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 0x55a508123180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b2f40f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a507ec8000 con 0x55a5082acc00 2024-09-13T23:43:27.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b0ef8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 0x55a508123180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b0ef8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 0x55a508123180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50794/0 (socket says 172.21.15.167:50794) 2024-09-13T23:43:27.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.267+0000 7f11b0ef8700 1 -- 172.21.15.167:0/1973011567 learned_addr learned my addr 172.21.15.167:0/1973011567 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11b0ef8700 1 -- 172.21.15.167:0/1973011567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a507e4cd00 con 0x55a5082acc00 2024-09-13T23:43:27.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11b0ef8700 1 --2- 172.21.15.167:0/1973011567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 0x55a508123180 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55a5082ba390 tx=0x55a5082bc300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11966c3700 1 -- 172.21.15.167:0/1973011567 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a5082c6160 con 0x55a5082acc00 2024-09-13T23:43:27.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11966c3700 1 -- 172.21.15.167:0/1973011567 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a5081fa300 con 0x55a5082acc00 2024-09-13T23:43:27.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11b2f40f40 1 -- 172.21.15.167:0/1973011567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a507e4c340 con 0x55a5082acc00 2024-09-13T23:43:27.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11966c3700 1 -- 172.21.15.167:0/1973011567 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a5082c62c0 con 0x55a5082acc00 2024-09-13T23:43:27.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11b2f40f40 1 -- 172.21.15.167:0/1973011567 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a507e4dd40 con 0x55a5082acc00 2024-09-13T23:43:27.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11966c3700 1 -- 172.21.15.167:0/1973011567 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55a507f35b80 con 0x55a5082acc00 2024-09-13T23:43:27.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11966c3700 1 --2- 172.21.15.167:0/1973011567 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a508338000 0x55a50833e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11966c3700 1 -- 172.21.15.167:0/1973011567 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a507ea1500 con 0x55a5082acc00 2024-09-13T23:43:27.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11b1efa700 1 --2- 172.21.15.167:0/1973011567 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a508338000 0x55a50833e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11b2f40f40 5 note: GC not initialized 2024-09-13T23:43:27.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.271+0000 7f11b1efa700 1 --2- 172.21.15.167:0/1973011567 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a508338000 0x55a50833e000 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55a508243c50 tx=0x55a5081a37a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.275+0000 7f11b2f40f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:27.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.275+0000 7f11b2f40f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:27.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.275+0000 7f1191eba700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:57 2024 2024-09-13T23:43:27.284 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:27.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.275+0000 7f1191698700 20 reqs_thread_entry: start 2024-09-13T23:43:27.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.275+0000 7f11b2f40f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.275+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5082ac000 con 0x55a5081ed000 2024-09-13T23:43:27.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.275+0000 7f1193ebe700 20 reqs_thread_entry: start 2024-09-13T23:43:27.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.275+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.275+0000 7f11b2f40f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:27.285 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:27.285 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:27.285 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:27.285 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:27.285 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:27.285 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:27.286 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:27.287 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:27.288 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:27.288 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:27.288 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:27.288 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:27.288 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:27.288 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:27.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.275+0000 7f11986c7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:27.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.275+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94167295008768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5082ac400 con 0x55a5081ed000 2024-09-13T23:43:27.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.279+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94167295008768] v33'63 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.279+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94167293328128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508172800 con 0x55a5081ed400 2024-09-13T23:43:27.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.283+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94167293328128] v33'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.283+0000 7f11b2f40f40 20 remove_watcher() i=0 2024-09-13T23:43:27.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.283+0000 7f11b2f40f40 2 removed watcher, disabling cache 2024-09-13T23:43:27.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.283+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94167293329280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508173000 con 0x55a5081ed000 2024-09-13T23:43:27.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.283+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94167293329280] v33'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a5081ed000 2024-09-13T23:43:27.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.283+0000 7f11b2f40f40 20 remove_watcher() i=1 2024-09-13T23:43:27.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.283+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94167293330432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a5081ed800 con 0x55a5081ed400 2024-09-13T23:43:27.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.287+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94167293330432] v33'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.287+0000 7f11b2f40f40 20 remove_watcher() i=2 2024-09-13T23:43:27.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.287+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94167293331584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508338400 con 0x55a508172400 2024-09-13T23:43:27.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.287+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94167293331584] v33'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.287+0000 7f11b2f40f40 20 remove_watcher() i=3 2024-09-13T23:43:27.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.287+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94167293332736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508338800 con 0x55a5081ed400 2024-09-13T23:43:27.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.291+0000 7f11b16f9700 1 -- 172.21.15.167:0/2605329865 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94167293332736] v33'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291200 con 0x55a5081ed400 2024-09-13T23:43:27.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.291+0000 7f11b2f40f40 20 remove_watcher() i=4 2024-09-13T23:43:27.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.291+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94167293333888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508338c00 con 0x55a508172000 2024-09-13T23:43:27.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.295+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94167293333888] v33'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.295+0000 7f11b2f40f40 20 remove_watcher() i=5 2024-09-13T23:43:27.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.295+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94167293335040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508339000 con 0x55a508172000 2024-09-13T23:43:27.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.295+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94167293335040] v33'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508291440 con 0x55a508172000 2024-09-13T23:43:27.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.295+0000 7f11b2f40f40 20 remove_watcher() i=6 2024-09-13T23:43:27.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.295+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94167293336192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a508339400 con 0x55a508172400 2024-09-13T23:43:27.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b1efa700 1 -- 172.21.15.167:0/2605329865 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94167293336192] v33'44 uv42 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a508290b40 con 0x55a508172400 2024-09-13T23:43:27.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 20 remove_watcher() i=7 2024-09-13T23:43:27.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a5081ed400 msgr2=0x55a508122100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a5081ed400 0x55a508122100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a5081ed000 msgr2=0x55a508121b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a5081ed000 0x55a508121b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a508172400 msgr2=0x55a508122c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a508172400 0x55a508122c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a508172000 msgr2=0x55a508122680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a508172000 0x55a508122680 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a5081ecc00 msgr2=0x55a508121600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a5081ecc00 0x55a508121600 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55a50811b920 tx=0x55a507f3f800 comp rx=0 tx=0).stop 2024-09-13T23:43:27.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 msgr2=0x55a508120b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b0ef8700 1 -- 172.21.15.167:0/2605329865 reap_dead start 2024-09-13T23:43:27.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 0x55a508120b00 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55a508cb2420 tx=0x55a5081a2840 comp rx=0 tx=0).stop 2024-09-13T23:43:27.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 shutdown_connections 2024-09-13T23:43:27.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/2605329865 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b800 0x55a508120b00 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 >> 172.21.15.167:0/2605329865 conn(0x55a507e3b400 msgr2=0x55a507f3b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 shutdown_connections 2024-09-13T23:43:27.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/2605329865 wait complete. 2024-09-13T23:43:27.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/1973011567 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a508338000 msgr2=0x55a50833e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/1973011567 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a508338000 0x55a50833e000 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55a508243c50 tx=0x55a5081a37a0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/1973011567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 msgr2=0x55a508123180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/1973011567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 0x55a508123180 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55a5082ba390 tx=0x55a5082bc300 comp rx=0 tx=0).stop 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/1973011567 shutdown_connections 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/1973011567 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a508338000 0x55a50833e000 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/1973011567 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a5082acc00 0x55a508123180 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/1973011567 >> 172.21.15.167:0/1973011567 conn(0x55a5082adc00 msgr2=0x55a507f3c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/1973011567 shutdown_connections 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.299+0000 7f11b2f40f40 1 -- 172.21.15.167:0/1973011567 wait complete. 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.303+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3300350869 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a5081ec000 msgr2=0x55a508121080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.303+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/3300350869 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a5081ec000 0x55a508121080 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55a508cb35f0 tx=0x55a5081a2240 comp rx=0 tx=0).stop 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.303+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3300350869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 msgr2=0x55a508120580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.303+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/3300350869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 0x55a508120580 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55a508c9faa0 tx=0x55a507f3fa40 comp rx=0 tx=0).stop 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.303+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3300350869 shutdown_connections 2024-09-13T23:43:27.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.303+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/3300350869 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a5081ec000 0x55a508121080 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.303+0000 7f11b2f40f40 1 --2- 172.21.15.167:0/3300350869 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a507e3b000 0x55a508120580 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.303+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3300350869 >> 172.21.15.167:0/3300350869 conn(0x55a507e3ac00 msgr2=0x55a507f3b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.303+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3300350869 shutdown_connections 2024-09-13T23:43:27.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.303+0000 7f11b2f40f40 1 -- 172.21.15.167:0/3300350869 wait complete. 2024-09-13T23:43:27.317 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:27.317 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-09-13T23:43:27.318 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-09-13T23:43:27.318 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key create --uid foo --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh 2024-09-13T23:43:27.365 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:27.365 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:27.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.355+0000 7fd21dd97f40 1 Processor -- start 2024-09-13T23:43:27.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.355+0000 7fd21dd97f40 1 -- start start 2024-09-13T23:43:27.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.355+0000 7fd21dd97f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f3373b9000 0x55f33769e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.355+0000 7fd21dd97f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f33747d500 con 0x55f3373b9400 2024-09-13T23:43:27.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.355+0000 7fd21dd97f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f337446a00 con 0x55f3373b9000 2024-09-13T23:43:27.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.355+0000 7fd21c550700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f3373b9000 0x55f33769e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.355+0000 7fd21c550700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f3373b9000 0x55f33769e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50800/0 (socket says 172.21.15.167:50800) 2024-09-13T23:43:27.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.355+0000 7fd21bd4f700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55f3373b9400 0x55f3374b8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45266/0 (socket says 172.21.15.167:45266) 2024-09-13T23:43:27.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.355+0000 7fd21c550700 1 -- 172.21.15.167:0/1854112475 learned_addr learned my addr 172.21.15.167:0/1854112475 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21c550700 1 -- 172.21.15.167:0/1854112475 >> v1:172.21.15.12:6789/0 conn(0x55f3373b9400 legacy=0x55f3374b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21c550700 1 -- 172.21.15.167:0/1854112475 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3373cbd40 con 0x55f3373b9000 2024-09-13T23:43:27.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21c550700 1 --2- 172.21.15.167:0/1854112475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9000 0x55f33769e580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55f337699a70 tx=0x55f3374bd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b2eda03f523b3e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21b54e700 1 -- 172.21.15.167:0/1854112475 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33745d4a0 con 0x55f3373b9000 2024-09-13T23:43:27.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21b54e700 1 -- 172.21.15.167:0/1854112475 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f33747d800 con 0x55f3373b9000 2024-09-13T23:43:27.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21b54e700 1 -- 172.21.15.167:0/1854112475 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33745d600 con 0x55f3373b9000 2024-09-13T23:43:27.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1854112475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9000 msgr2=0x55f33769e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/1854112475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9000 0x55f33769e580 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55f337699a70 tx=0x55f3374bd920 comp rx=0 tx=0).stop 2024-09-13T23:43:27.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1854112475 shutdown_connections 2024-09-13T23:43:27.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/1854112475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9000 0x55f33769e580 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1854112475 >> 172.21.15.167:0/1854112475 conn(0x55f3373b8c00 msgr2=0x55f3374b8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1854112475 shutdown_connections 2024-09-13T23:43:27.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1854112475 wait complete. 2024-09-13T23:43:27.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21dd97f40 1 Processor -- start 2024-09-13T23:43:27.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21dd97f40 1 -- start start 2024-09-13T23:43:27.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21dd97f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9000 0x55f33769e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21dd97f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f337446500 con 0x55f3373b9000 2024-09-13T23:43:27.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21c550700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9000 0x55f33769e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21c550700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9000 0x55f33769e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50814/0 (socket says 172.21.15.167:50814) 2024-09-13T23:43:27.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21c550700 1 -- 172.21.15.167:0/3177698082 learned_addr learned my addr 172.21.15.167:0/3177698082 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.359+0000 7fd21c550700 1 -- 172.21.15.167:0/3177698082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3373cbba0 con 0x55f3373b9000 2024-09-13T23:43:27.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21c550700 1 --2- 172.21.15.167:0/3177698082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9000 0x55f33769e580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55f337698de0 tx=0x55f3374bd800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9063f646e3ede52d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21ad4d700 1 -- 172.21.15.167:0/3177698082 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33745d760 con 0x55f3373b9000 2024-09-13T23:43:27.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21ad4d700 1 -- 172.21.15.167:0/3177698082 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f33747db00 con 0x55f3373b9000 2024-09-13T23:43:27.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21ad4d700 1 -- 172.21.15.167:0/3177698082 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33745d4a0 con 0x55f3373b9000 2024-09-13T23:43:27.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21dd97f40 1 -- 172.21.15.167:0/3177698082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9000 msgr2=0x55f33769e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/3177698082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9000 0x55f33769e580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55f337698de0 tx=0x55f3374bd800 comp rx=0 tx=0).stop 2024-09-13T23:43:27.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21dd97f40 1 -- 172.21.15.167:0/3177698082 shutdown_connections 2024-09-13T23:43:27.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/3177698082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9000 0x55f33769e580 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21dd97f40 1 -- 172.21.15.167:0/3177698082 >> 172.21.15.167:0/3177698082 conn(0x55f3373b8c00 msgr2=0x55f3374b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21dd97f40 1 -- 172.21.15.167:0/3177698082 shutdown_connections 2024-09-13T23:43:27.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21dd97f40 1 -- 172.21.15.167:0/3177698082 wait complete. 2024-09-13T23:43:27.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21dd97f40 1 Processor -- start 2024-09-13T23:43:27.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21dd97f40 1 -- start start 2024-09-13T23:43:27.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21dd97f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9400 0x55f33769eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21dd97f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3374463c0 con 0x55f3373b9400 2024-09-13T23:43:27.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21c550700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9400 0x55f33769eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21c550700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9400 0x55f33769eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50816/0 (socket says 172.21.15.167:50816) 2024-09-13T23:43:27.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21c550700 1 -- 172.21.15.167:0/1763799517 learned_addr learned my addr 172.21.15.167:0/1763799517 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21c550700 1 -- 172.21.15.167:0/1763799517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3373cba00 con 0x55f3373b9400 2024-09-13T23:43:27.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21c550700 1 --2- 172.21.15.167:0/1763799517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9400 0x55f33769eb00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55f337515e90 tx=0x55f3374bd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.363+0000 7fd21954a700 1 -- 172.21.15.167:0/1763799517 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33745d600 con 0x55f3373b9400 2024-09-13T23:43:27.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21954a700 1 -- 172.21.15.167:0/1763799517 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f33747dc80 con 0x55f3373b9400 2024-09-13T23:43:27.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1763799517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f3373cb860 con 0x55f3373b9400 2024-09-13T23:43:27.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21954a700 1 -- 172.21.15.167:0/1763799517 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33745d8c0 con 0x55f3373b9400 2024-09-13T23:43:27.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1763799517 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3373cb6c0 con 0x55f3373b9400 2024-09-13T23:43:27.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21dd97f40 1 Processor -- start 2024-09-13T23:43:27.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21dd97f40 1 -- start start 2024-09-13T23:43:27.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21954a700 1 -- 172.21.15.167:0/1763799517 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f3374b2b00 con 0x55f3373b9400 2024-09-13T23:43:27.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21954a700 1 --2- 172.21.15.167:0/1763799517 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3373b9c00 0x55f33769f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21954a700 1 -- 172.21.15.167:0/1763799517 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f33741f180 con 0x55f3373b9400 2024-09-13T23:43:27.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21dd97f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 0x55f33769f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21dd97f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3373de640 con 0x55f33775a000 2024-09-13T23:43:27.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21cd51700 1 --2- 172.21.15.167:0/1763799517 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3373b9c00 0x55f33769f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21c550700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 0x55f33769f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21c550700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 0x55f33769f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50832/0 (socket says 172.21.15.167:50832) 2024-09-13T23:43:27.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21c550700 1 -- 172.21.15.167:0/3636411438 learned_addr learned my addr 172.21.15.167:0/3636411438 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21c550700 1 -- 172.21.15.167:0/3636411438 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3373ca000 con 0x55f33775a000 2024-09-13T23:43:27.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21cd51700 1 --2- 172.21.15.167:0/1763799517 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3373b9c00 0x55f33769f080 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55f3376dc1e0 tx=0x55f33771cb40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.367+0000 7fd21c550700 1 --2- 172.21.15.167:0/3636411438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 0x55f33769f600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55f3376dc270 tx=0x55f33771cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162e1f8f837be662 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd216d45700 1 -- 172.21.15.167:0/3636411438 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33745d600 con 0x55f33775a000 2024-09-13T23:43:27.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd216d45700 1 -- 172.21.15.167:0/3636411438 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f33747dc80 con 0x55f33775a000 2024-09-13T23:43:27.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd216d45700 1 -- 172.21.15.167:0/3636411438 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33745d8c0 con 0x55f33775a000 2024-09-13T23:43:27.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 -- 172.21.15.167:0/3636411438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 msgr2=0x55f33769f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/3636411438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 0x55f33769f600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55f3376dc270 tx=0x55f33771cc60 comp rx=0 tx=0).stop 2024-09-13T23:43:27.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 -- 172.21.15.167:0/3636411438 shutdown_connections 2024-09-13T23:43:27.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/3636411438 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 0x55f33769f600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 -- 172.21.15.167:0/3636411438 >> 172.21.15.167:0/3636411438 conn(0x55f3373b9800 msgr2=0x55f3374b9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 -- 172.21.15.167:0/3636411438 shutdown_connections 2024-09-13T23:43:27.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 -- 172.21.15.167:0/3636411438 wait complete. 2024-09-13T23:43:27.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 Processor -- start 2024-09-13T23:43:27.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 -- start start 2024-09-13T23:43:27.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 0x55f33769f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3373de8c0 con 0x55f33775a000 2024-09-13T23:43:27.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21c550700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 0x55f33769f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21c550700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 0x55f33769f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50834/0 (socket says 172.21.15.167:50834) 2024-09-13T23:43:27.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 learned_addr learned my addr 172.21.15.167:0/122050225 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3373caea0 con 0x55f33775a000 2024-09-13T23:43:27.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21c550700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 0x55f33769f600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55f3376dc510 tx=0x55f33771ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd215542700 1 -- 172.21.15.167:0/122050225 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33745d760 con 0x55f33775a000 2024-09-13T23:43:27.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f3373ca820 con 0x55f33775a000 2024-09-13T23:43:27.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd215542700 1 -- 172.21.15.167:0/122050225 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f33777c000 con 0x55f33775a000 2024-09-13T23:43:27.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd215542700 1 -- 172.21.15.167:0/122050225 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33745d4a0 con 0x55f33775a000 2024-09-13T23:43:27.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.371+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3373ca9c0 con 0x55f33775a000 2024-09-13T23:43:27.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd215542700 1 -- 172.21.15.167:0/122050225 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f3374b2dc0 con 0x55f33775a000 2024-09-13T23:43:27.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd215542700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f33775b800 0x55f3376a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd215542700 1 -- 172.21.15.167:0/122050225 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f33741f340 con 0x55f33775a000 2024-09-13T23:43:27.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21bd4f700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f33775b800 0x55f3376a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f33775a400 0x55f33769fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f33775a800 con 0x55f33775a400 2024-09-13T23:43:27.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21cd51700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f33775a400 0x55f33769fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21bd4f700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f33775b800 0x55f3376a0680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55f337699f80 tx=0x55f3377f0000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21cd51700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f33775a400 0x55f33769fb80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21dd97f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21dd97f40 20 realm 2024-09-13T23:43:27.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f33775ac00 con 0x55f33775a400 2024-09-13T23:43:27.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21dd97f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21dd97f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:27.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.375+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f33775a800 con 0x55f33775a400 2024-09-13T23:43:27.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21dd97f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f33775ac00 0x55f3376a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f33775bc00 con 0x55f33775ac00 2024-09-13T23:43:27.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21c550700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f33775ac00 0x55f3376a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21c550700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f33775ac00 0x55f3376a0100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21dd97f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:27.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc000 con 0x55f33775ac00 2024-09-13T23:43:27.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21dd97f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:27.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21dd97f40 20 searching for the correct realm 2024-09-13T23:43:27.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f3377fc400 0x55f3376a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc400 2024-09-13T23:43:27.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21bd4f700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f3377fc400 0x55f3376a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.379+0000 7fd21bd4f700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f3377fc400 0x55f3376a0c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775ac00 2024-09-13T23:43:27.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21c550700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f3377fc000 0x55f33769e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc000 2024-09-13T23:43:27.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21cd51700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f3377fc000 0x55f33769e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21cd51700 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f3377fc000 0x55f33769e580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775a400 2024-09-13T23:43:27.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775a400 2024-09-13T23:43:27.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.383+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc400 2024-09-13T23:43:27.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775a400 2024-09-13T23:43:27.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775ac00 2024-09-13T23:43:27.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775ac00 2024-09-13T23:43:27.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775a400 2024-09-13T23:43:27.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc000 2024-09-13T23:43:27.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775a400 2024-09-13T23:43:27.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.387+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc000 2024-09-13T23:43:27.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.391+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.391+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775ac00 2024-09-13T23:43:27.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.391+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.391+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc400 2024-09-13T23:43:27.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.391+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.391+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775a400 2024-09-13T23:43:27.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.391+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.391+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775ac00 2024-09-13T23:43:27.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.391+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.391+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc400 2024-09-13T23:43:27.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.391+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.391+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775a400 2024-09-13T23:43:27.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775a400 2024-09-13T23:43:27.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775a400 2024-09-13T23:43:27.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc400 2024-09-13T23:43:27.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc000 2024-09-13T23:43:27.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775a400 2024-09-13T23:43:27.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc000 2024-09-13T23:43:27.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.395+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc400 2024-09-13T23:43:27.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.399+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.399+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc000 2024-09-13T23:43:27.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.399+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.399+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775ac00 2024-09-13T23:43:27.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.399+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.399+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775ac00 2024-09-13T23:43:27.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.399+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.399+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f3377fc400 2024-09-13T23:43:27.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.399+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.399+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775ac00 2024-09-13T23:43:27.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775ac00 2024-09-13T23:43:27.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:27.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:27.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:27.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:27.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fcc00 con 0x55f33775a400 2024-09-13T23:43:27.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fd000 con 0x55f33775ac00 2024-09-13T23:43:27.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:27.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fd400 con 0x55f33775ac00 2024-09-13T23:43:27.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:27.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 zone default found 2024-09-13T23:43:27.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 4 Realm: () 2024-09-13T23:43:27.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:27.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:27.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:27.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 zonegroup default 2024-09-13T23:43:27.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fd800 con 0x55f3377fc000 2024-09-13T23:43:27.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.403+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fdc00 con 0x55f3377fc400 2024-09-13T23:43:27.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.407+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.407+0000 7fd21dd97f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.407+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.407+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fcc00 con 0x55f3377fc400 2024-09-13T23:43:27.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.407+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.407+0000 7fd21dd97f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.407+0000 7fd21dd97f40 20 started sync module instance, tier type = 2024-09-13T23:43:27.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.407+0000 7fd21dd97f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:27.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.407+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fd800 con 0x55f33775ac00 2024-09-13T23:43:27.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.411+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.411+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94503095065344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f33775a800 con 0x55f33775ac00 2024-09-13T23:43:27.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.411+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775a400 2024-09-13T23:43:27.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.411+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94503095065344] v33'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.411+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.411+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94503095066496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fd000 con 0x55f33775a400 2024-09-13T23:43:27.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.411+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fd400 con 0x55f33775ac00 2024-09-13T23:43:27.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.415+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94503095066496] v33'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.415+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.415+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94503095067648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fd800 con 0x55f33775ac00 2024-09-13T23:43:27.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.415+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fdc00 con 0x55f3377fc000 2024-09-13T23:43:27.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.419+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.419+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94503095068800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fcc00 con 0x55f3377fc000 2024-09-13T23:43:27.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.419+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f33775bc00 con 0x55f33775ac00 2024-09-13T23:43:27.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.419+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94503095067648] v33'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.419+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94503095068800] v33'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.419+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.419+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94503095069952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fc800 con 0x55f33775ac00 2024-09-13T23:43:27.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.419+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fd000 con 0x55f3377fc400 2024-09-13T23:43:27.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.423+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.423+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94503095071104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fdc00 con 0x55f3377fc400 2024-09-13T23:43:27.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.423+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fcc00 con 0x55f3377fc400 2024-09-13T23:43:27.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.423+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94503095069952] v33'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.423+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94503095071104] v33'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94503095072256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fd000 con 0x55f3377fc400 2024-09-13T23:43:27.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fdc00 con 0x55f3377fc000 2024-09-13T23:43:27.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94503095072256] v33'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'45 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94503095073408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fcc00 con 0x55f3377fc000 2024-09-13T23:43:27.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21dd97f40 20 add_watcher() i=0 2024-09-13T23:43:27.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21dd97f40 20 add_watcher() i=1 2024-09-13T23:43:27.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21dd97f40 20 add_watcher() i=2 2024-09-13T23:43:27.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21dd97f40 20 add_watcher() i=3 2024-09-13T23:43:27.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21dd97f40 20 add_watcher() i=4 2024-09-13T23:43:27.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21dd97f40 20 add_watcher() i=5 2024-09-13T23:43:27.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.427+0000 7fd21dd97f40 20 add_watcher() i=6 2024-09-13T23:43:27.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.431+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94503095073408] v33'46 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.431+0000 7fd21dd97f40 20 add_watcher() i=7 2024-09-13T23:43:27.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.431+0000 7fd21dd97f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:27.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.431+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337808000 con 0x55f33775a400 2024-09-13T23:43:27.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.431+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.431+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94503096598528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337808400 con 0x55f33775a400 2024-09-13T23:43:27.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.431+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94503096598528] v33'64 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.431+0000 7fd20351e700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:27.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.431+0000 7fd21dd97f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:27.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.431+0000 7fd21dd97f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:27.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.431+0000 7fd21dd97f40 1 Processor -- start 2024-09-13T23:43:27.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 -- start start 2024-09-13T23:43:27.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 0x55f3376a1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3374463c0 con 0x55f337808c00 2024-09-13T23:43:27.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21bd4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 0x55f3376a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21bd4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 0x55f3376a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50840/0 (socket says 172.21.15.167:50840) 2024-09-13T23:43:27.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21bd4f700 1 -- 172.21.15.167:0/1733179697 learned_addr learned my addr 172.21.15.167:0/1733179697 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21bd4f700 1 -- 172.21.15.167:0/1733179697 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3373cab60 con 0x55f337808c00 2024-09-13T23:43:27.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21bd4f700 1 --2- 172.21.15.167:0/1733179697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 0x55f3376a1180 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55f337814060 tx=0x55f33771df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608fd74e04c2d1d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd202d1d700 1 -- 172.21.15.167:0/1733179697 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33780a160 con 0x55f337808c00 2024-09-13T23:43:27.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd202d1d700 1 -- 172.21.15.167:0/1733179697 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f33777ca80 con 0x55f337808c00 2024-09-13T23:43:27.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd202d1d700 1 -- 172.21.15.167:0/1733179697 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33780a2c0 con 0x55f337808c00 2024-09-13T23:43:27.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1733179697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 msgr2=0x55f3376a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/1733179697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 0x55f3376a1180 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55f337814060 tx=0x55f33771df20 comp rx=0 tx=0).stop 2024-09-13T23:43:27.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1733179697 shutdown_connections 2024-09-13T23:43:27.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/1733179697 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 0x55f3376a1180 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1733179697 >> 172.21.15.167:0/1733179697 conn(0x55f337808800 msgr2=0x55f3374ba000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1733179697 shutdown_connections 2024-09-13T23:43:27.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1733179697 wait complete. 2024-09-13T23:43:27.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 Processor -- start 2024-09-13T23:43:27.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 -- start start 2024-09-13T23:43:27.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 0x55f3376a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.435+0000 7fd21dd97f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f337446500 con 0x55f337808c00 2024-09-13T23:43:27.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd21bd4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 0x55f3376a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd21bd4f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 0x55f3376a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50850/0 (socket says 172.21.15.167:50850) 2024-09-13T23:43:27.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd21bd4f700 1 -- 172.21.15.167:0/1851913499 learned_addr learned my addr 172.21.15.167:0/1851913499 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd21bd4f700 1 -- 172.21.15.167:0/1851913499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3373cad00 con 0x55f337808c00 2024-09-13T23:43:27.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd21bd4f700 1 --2- 172.21.15.167:0/1851913499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 0x55f3376a1180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55f337815320 tx=0x55f3377f00c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd20151a700 1 -- 172.21.15.167:0/1851913499 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33780a420 con 0x55f337808c00 2024-09-13T23:43:27.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1851913499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f3373ca340 con 0x55f337808c00 2024-09-13T23:43:27.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd20151a700 1 -- 172.21.15.167:0/1851913499 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f33777c780 con 0x55f337808c00 2024-09-13T23:43:27.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd20151a700 1 -- 172.21.15.167:0/1851913499 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f33780a160 con 0x55f337808c00 2024-09-13T23:43:27.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1851913499 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3373cab60 con 0x55f337808c00 2024-09-13T23:43:27.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd20151a700 1 -- 172.21.15.167:0/1851913499 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f3374b38c0 con 0x55f337808c00 2024-09-13T23:43:27.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd20151a700 1 --2- 172.21.15.167:0/1851913499 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f337894000 0x55f33789a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd20151a700 1 -- 172.21.15.167:0/1851913499 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f33741f6c0 con 0x55f337808c00 2024-09-13T23:43:27.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd21dd97f40 5 note: GC not initialized 2024-09-13T23:43:27.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.439+0000 7fd21cd51700 1 --2- 172.21.15.167:0/1851913499 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f337894000 0x55f33789a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21cd51700 1 --2- 172.21.15.167:0/1851913499 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f337894000 0x55f33789a000 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55f337799ad0 tx=0x55f33771ca80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd1fed15700 20 reqs_thread_entry: start 2024-09-13T23:43:27.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21dd97f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21dd97f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd1fcd11700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:57 2024 2024-09-13T23:43:27.453 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21dd97f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337809800 con 0x55f33775a400 2024-09-13T23:43:27.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd1fc4ef700 20 reqs_thread_entry: start 2024-09-13T23:43:27.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21dd97f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21dd97f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21dd97f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21dd97f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21dd97f40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-09-13T23:43:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21dd97f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.443+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337809c00 con 0x55f3377fc400 2024-09-13T23:43:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21dd97f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21dd97f40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-09-13T23:43:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21dd97f40 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-13T23:43:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21dd97f40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-13T23:43:27.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~459 in=459b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337894800 con 0x55f33775a400 2024-09-13T23:43:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~459] v33'4 uv4 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21dd97f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21dd97f40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21dd97f40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:27.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94503096600832 in=647b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337894c00 con 0x55f33775ac00 2024-09-13T23:43:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94503095069952 notify 141733920771 ret 0) v3 ==== 677+0+0 (crc 0 0 0) 0x55f3373cb6c0 con 0x55f33775ac00 2024-09-13T23:43:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 notify.4 [notify cookie 94503096600832 out=8b] v0'0 uv45 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd216d45700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920771 cookie 94503095069952 notifier 4350 bl.length()=635 2024-09-13T23:43:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd216d45700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd216d45700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:27.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.447+0000 7fd216d45700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f33775a800 con 0x55f33775ac00 2024-09-13T23:43:27.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.451+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.451+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94503096600832 notify 141733920771 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f3373cbd40 con 0x55f33775ac00 2024-09-13T23:43:27.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.451+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337895000 con 0x55f3377fc400 2024-09-13T23:43:27.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.451+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v33'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.451+0000 7fd21dd97f40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-13T23:43:27.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.451+0000 7fd21dd97f40 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-13T23:43:27.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.451+0000 7fd21dd97f40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-13T23:43:27.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.451+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94503096600832 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337895400 con 0x55f33775ac00 2024-09-13T23:43:27.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.455+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== watch-notify(notify (1) cookie 94503095067648 notify 141733920772 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55f3373cbd40 con 0x55f33775ac00 2024-09-13T23:43:27.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.455+0000 7fd216d45700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920772 cookie 94503095067648 notifier 4350 bl.length()=177 2024-09-13T23:43:27.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.455+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(68 notify.2 [notify cookie 94503096600832 out=8b] v0'0 uv45 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.455+0000 7fd216d45700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-13T23:43:27.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.455+0000 7fd216d45700 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-13T23:43:27.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.455+0000 7fd216d45700 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f33775bc00 con 0x55f33775ac00 2024-09-13T23:43:27.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.455+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.455+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== watch-notify(notify_complete (2) cookie 94503096600832 notify 141733920772 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f3373cbba0 con 0x55f33775ac00 2024-09-13T23:43:27.464 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:27.464 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:27.464 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:27.464 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:27.464 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:27.464 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:27.465 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:27.466 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:27.467 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:27.467 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:27.467 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:27.467 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:27.467 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:27.467 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:27.467 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:27.467 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:27.467 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:27.467 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:27.467 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:27.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.455+0000 7fd20351e700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:27.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.455+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94503096598528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337895800 con 0x55f33775a400 2024-09-13T23:43:27.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.459+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94503096598528] v33'65 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.459+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94503095065344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337895c00 con 0x55f33775ac00 2024-09-13T23:43:27.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.459+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94503095065344] v33'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.459+0000 7fd21dd97f40 20 remove_watcher() i=0 2024-09-13T23:43:27.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.459+0000 7fd21dd97f40 2 removed watcher, disabling cache 2024-09-13T23:43:27.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.459+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94503095066496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337808000 con 0x55f33775a400 2024-09-13T23:43:27.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.463+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94503095066496] v33'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f33775a400 2024-09-13T23:43:27.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.463+0000 7fd21dd97f40 20 remove_watcher() i=1 2024-09-13T23:43:27.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.463+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94503095067648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337808400 con 0x55f33775ac00 2024-09-13T23:43:27.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.467+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94503095067648] v33'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.467+0000 7fd21dd97f40 20 remove_watcher() i=2 2024-09-13T23:43:27.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.467+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94503095068800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337809800 con 0x55f3377fc000 2024-09-13T23:43:27.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.467+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94503095068800] v33'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.467+0000 7fd21dd97f40 20 remove_watcher() i=3 2024-09-13T23:43:27.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.467+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94503095069952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fdc00 con 0x55f33775ac00 2024-09-13T23:43:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.471+0000 7fd21c550700 1 -- 172.21.15.167:0/122050225 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94503095069952] v33'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1200 con 0x55f33775ac00 2024-09-13T23:43:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.471+0000 7fd21dd97f40 20 remove_watcher() i=4 2024-09-13T23:43:27.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.471+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94503095071104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fcc00 con 0x55f3377fc400 2024-09-13T23:43:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.475+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94503095071104] v33'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.475+0000 7fd21dd97f40 20 remove_watcher() i=5 2024-09-13T23:43:27.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.475+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94503095072256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f337894c00 con 0x55f3377fc400 2024-09-13T23:43:27.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.475+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94503095072256] v33'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e1440 con 0x55f3377fc400 2024-09-13T23:43:27.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.475+0000 7fd21dd97f40 20 remove_watcher() i=6 2024-09-13T23:43:27.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.475+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94503095073408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3377fd400 con 0x55f3377fc000 2024-09-13T23:43:27.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21cd51700 1 -- 172.21.15.167:0/122050225 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94503095073408] v33'47 uv45 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f3377e0b40 con 0x55f3377fc000 2024-09-13T23:43:27.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 20 remove_watcher() i=7 2024-09-13T23:43:27.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f33775ac00 msgr2=0x55f3376a0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f33775ac00 0x55f3376a0100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f33775a400 msgr2=0x55f33769fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f33775a400 0x55f33769fb80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f3377fc000 msgr2=0x55f33769e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f3377fc000 0x55f33769e580 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f3377fc400 msgr2=0x55f3376a0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f3377fc400 0x55f3376a0c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f33775b800 msgr2=0x55f3376a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f33775b800 0x55f3376a0680 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55f337699f80 tx=0x55f3377f0000 comp rx=0 tx=0).stop 2024-09-13T23:43:27.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 msgr2=0x55f33769f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 0x55f33769f600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55f3376dc510 tx=0x55f33771ce40 comp rx=0 tx=0).stop 2024-09-13T23:43:27.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21bd4f700 1 -- 172.21.15.167:0/122050225 reap_dead start 2024-09-13T23:43:27.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 shutdown_connections 2024-09-13T23:43:27.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/122050225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f33775a000 0x55f33769f600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 >> 172.21.15.167:0/122050225 conn(0x55f3373b9800 msgr2=0x55f3374b9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 shutdown_connections 2024-09-13T23:43:27.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/122050225 wait complete. 2024-09-13T23:43:27.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1851913499 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f337894000 msgr2=0x55f33789a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/1851913499 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f337894000 0x55f33789a000 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55f337799ad0 tx=0x55f33771ca80 comp rx=0 tx=0).stop 2024-09-13T23:43:27.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1851913499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 msgr2=0x55f3376a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/1851913499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 0x55f3376a1180 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x55f337815320 tx=0x55f3377f00c0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1851913499 shutdown_connections 2024-09-13T23:43:27.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/1851913499 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f337894000 0x55f33789a000 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/1851913499 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f337808c00 0x55f3376a1180 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1851913499 >> 172.21.15.167:0/1851913499 conn(0x55f337808800 msgr2=0x55f3374ba000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1851913499 shutdown_connections 2024-09-13T23:43:27.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1851913499 wait complete. 2024-09-13T23:43:27.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1763799517 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3373b9c00 msgr2=0x55f33769f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/1763799517 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3373b9c00 0x55f33769f080 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55f3376dc1e0 tx=0x55f33771cb40 comp rx=0 tx=0).stop 2024-09-13T23:43:27.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1763799517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9400 msgr2=0x55f33769eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/1763799517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9400 0x55f33769eb00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x55f337515e90 tx=0x55f3374bd920 comp rx=0 tx=0).stop 2024-09-13T23:43:27.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1763799517 shutdown_connections 2024-09-13T23:43:27.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/1763799517 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3373b9c00 0x55f33769f080 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 --2- 172.21.15.167:0/1763799517 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f3373b9400 0x55f33769eb00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1763799517 >> 172.21.15.167:0/1763799517 conn(0x55f3373b8c00 msgr2=0x55f3374b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1763799517 shutdown_connections 2024-09-13T23:43:27.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.479+0000 7fd21dd97f40 1 -- 172.21.15.167:0/1763799517 wait complete. 2024-09-13T23:43:27.494 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:27.495 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-13T23:43:27.495 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-13T23:43:27.495 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-13T23:43:27.534 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:27.534 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:27.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd749cfaf40 1 Processor -- start 2024-09-13T23:43:27.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd749cfaf40 1 -- start start 2024-09-13T23:43:27.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd749cfaf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x557e6558d000 0x557e65872580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd749cfaf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x557e65651500 con 0x557e6558d400 2024-09-13T23:43:27.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd749cfaf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x557e6561aa00 con 0x557e6558d000 2024-09-13T23:43:27.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd7484b3700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x557e6558d000 0x557e65872580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd747cb2700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x557e6558d400 0x557e6568c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45276/0 (socket says 172.21.15.167:45276) 2024-09-13T23:43:27.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd747cb2700 1 -- 172.21.15.167:0/1677318929 learned_addr learned my addr 172.21.15.167:0/1677318929 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:27.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd7484b3700 1 -- 172.21.15.167:0/1677318929 >> v1:172.21.15.12:6789/0 conn(0x557e6558d400 legacy=0x557e6568c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd7484b3700 1 -- 172.21.15.167:0/1677318929 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6559fd40 con 0x557e6558d000 2024-09-13T23:43:27.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd7484b3700 1 --2- 172.21.15.167:0/1677318929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x557e6586da70 tx=0x557e65691920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c26554680b87ae0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd7474b1700 1 -- 172.21.15.167:0/1677318929 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e656314a0 con 0x557e6558d000 2024-09-13T23:43:27.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd7474b1700 1 -- 172.21.15.167:0/1677318929 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557e65651800 con 0x557e6558d000 2024-09-13T23:43:27.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd7474b1700 1 -- 172.21.15.167:0/1677318929 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e65631600 con 0x557e6558d000 2024-09-13T23:43:27.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd749cfaf40 1 -- 172.21.15.167:0/1677318929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 msgr2=0x557e65872580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/1677318929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x557e6586da70 tx=0x557e65691920 comp rx=0 tx=0).stop 2024-09-13T23:43:27.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd749cfaf40 1 -- 172.21.15.167:0/1677318929 shutdown_connections 2024-09-13T23:43:27.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/1677318929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.527+0000 7fd749cfaf40 1 -- 172.21.15.167:0/1677318929 >> 172.21.15.167:0/1677318929 conn(0x557e6558cc00 msgr2=0x557e6568c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 -- 172.21.15.167:0/1677318929 shutdown_connections 2024-09-13T23:43:27.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 -- 172.21.15.167:0/1677318929 wait complete. 2024-09-13T23:43:27.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 Processor -- start 2024-09-13T23:43:27.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 -- start start 2024-09-13T23:43:27.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6561a500 con 0x557e6558d000 2024-09-13T23:43:27.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd7484b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd7484b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50866/0 (socket says 172.21.15.167:50866) 2024-09-13T23:43:27.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd7484b3700 1 -- 172.21.15.167:0/2705104446 learned_addr learned my addr 172.21.15.167:0/2705104446 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd7484b3700 1 -- 172.21.15.167:0/2705104446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6559fba0 con 0x557e6558d000 2024-09-13T23:43:27.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd7484b3700 1 --2- 172.21.15.167:0/2705104446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x557e663f1800 tx=0x557e65691800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816a1e1ab6169373 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd746cb0700 1 -- 172.21.15.167:0/2705104446 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e65631760 con 0x557e6558d000 2024-09-13T23:43:27.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd746cb0700 1 -- 172.21.15.167:0/2705104446 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557e65651b00 con 0x557e6558d000 2024-09-13T23:43:27.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd746cb0700 1 -- 172.21.15.167:0/2705104446 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e656314a0 con 0x557e6558d000 2024-09-13T23:43:27.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2705104446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 msgr2=0x557e65872580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2705104446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x557e663f1800 tx=0x557e65691800 comp rx=0 tx=0).stop 2024-09-13T23:43:27.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2705104446 shutdown_connections 2024-09-13T23:43:27.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2705104446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2705104446 >> 172.21.15.167:0/2705104446 conn(0x557e6558cc00 msgr2=0x557e6568d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2705104446 shutdown_connections 2024-09-13T23:43:27.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2705104446 wait complete. 2024-09-13T23:43:27.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 Processor -- start 2024-09-13T23:43:27.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 -- start start 2024-09-13T23:43:27.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.531+0000 7fd749cfaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6561a3c0 con 0x557e6558d000 2024-09-13T23:43:27.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd7484b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd7484b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50882/0 (socket says 172.21.15.167:50882) 2024-09-13T23:43:27.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd7484b3700 1 -- 172.21.15.167:0/3153849412 learned_addr learned my addr 172.21.15.167:0/3153849412 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd7484b3700 1 -- 172.21.15.167:0/3153849412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6559fa00 con 0x557e6558d000 2024-09-13T23:43:27.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd7484b3700 1 --2- 172.21.15.167:0/3153849412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x557e663f16b0 tx=0x557e65691920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd7454ad700 1 -- 172.21.15.167:0/3153849412 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e65631600 con 0x557e6558d000 2024-09-13T23:43:27.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd749cfaf40 1 -- 172.21.15.167:0/3153849412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557e6559f860 con 0x557e6558d000 2024-09-13T23:43:27.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd7454ad700 1 -- 172.21.15.167:0/3153849412 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557e65651c80 con 0x557e6558d000 2024-09-13T23:43:27.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd7454ad700 1 -- 172.21.15.167:0/3153849412 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e656318c0 con 0x557e6558d000 2024-09-13T23:43:27.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd749cfaf40 1 -- 172.21.15.167:0/3153849412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e6559f6c0 con 0x557e6558d000 2024-09-13T23:43:27.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd749cfaf40 1 Processor -- start 2024-09-13T23:43:27.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd749cfaf40 1 -- start start 2024-09-13T23:43:27.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd7454ad700 1 -- 172.21.15.167:0/3153849412 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x557e65686b00 con 0x557e6558d000 2024-09-13T23:43:27.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd749cfaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 0x557e65872b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd749cfaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6561a000 con 0x557e6558d800 2024-09-13T23:43:27.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd7454ad700 1 --2- 172.21.15.167:0/3153849412 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65928000 0x557e65873080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd7454ad700 1 -- 172.21.15.167:0/3153849412 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x557e655f3180 con 0x557e6558d000 2024-09-13T23:43:27.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.535+0000 7fd748cb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 0x557e65872b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd748cb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 0x557e65872b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50890/0 (socket says 172.21.15.167:50890) 2024-09-13T23:43:27.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd748cb4700 1 -- 172.21.15.167:0/2399945958 learned_addr learned my addr 172.21.15.167:0/2399945958 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd7484b3700 1 --2- 172.21.15.167:0/3153849412 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65928000 0x557e65873080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd748cb4700 1 -- 172.21.15.167:0/2399945958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6559e000 con 0x557e6558d800 2024-09-13T23:43:27.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd7484b3700 1 --2- 172.21.15.167:0/3153849412 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65928000 0x557e65873080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x557e658b4f60 tx=0x557e658f0b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd748cb4700 1 --2- 172.21.15.167:0/2399945958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 0x557e65872b00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x557e658b4240 tx=0x557e658f0e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e655793d51c30108 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd742ca8700 1 -- 172.21.15.167:0/2399945958 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e65631600 con 0x557e6558d800 2024-09-13T23:43:27.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd742ca8700 1 -- 172.21.15.167:0/2399945958 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557e6593a000 con 0x557e6558d800 2024-09-13T23:43:27.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd742ca8700 1 -- 172.21.15.167:0/2399945958 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e656318c0 con 0x557e6558d800 2024-09-13T23:43:27.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2399945958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 msgr2=0x557e65872b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2399945958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 0x557e65872b00 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x557e658b4240 tx=0x557e658f0e40 comp rx=0 tx=0).stop 2024-09-13T23:43:27.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2399945958 shutdown_connections 2024-09-13T23:43:27.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2399945958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 0x557e65872b00 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2399945958 >> 172.21.15.167:0/2399945958 conn(0x557e6558d400 msgr2=0x557e6568d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2399945958 shutdown_connections 2024-09-13T23:43:27.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2399945958 wait complete. 2024-09-13T23:43:27.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd749cfaf40 1 Processor -- start 2024-09-13T23:43:27.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd749cfaf40 1 -- start start 2024-09-13T23:43:27.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd749cfaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 0x557e65872b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.539+0000 7fd749cfaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e655b2640 con 0x557e6558d800 2024-09-13T23:43:27.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd748cb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 0x557e65872b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd748cb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 0x557e65872b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50906/0 (socket says 172.21.15.167:50906) 2024-09-13T23:43:27.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 learned_addr learned my addr 172.21.15.167:0/2864844210 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6559eea0 con 0x557e6558d800 2024-09-13T23:43:27.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd748cb4700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 0x557e65872b00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x557e663f1950 tx=0x557e658f0a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd7414a5700 1 -- 172.21.15.167:0/2864844210 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e65631760 con 0x557e6558d800 2024-09-13T23:43:27.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd7414a5700 1 -- 172.21.15.167:0/2864844210 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557e6593a180 con 0x557e6558d800 2024-09-13T23:43:27.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557e6559e820 con 0x557e6558d800 2024-09-13T23:43:27.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd7414a5700 1 -- 172.21.15.167:0/2864844210 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e65631600 con 0x557e6558d800 2024-09-13T23:43:27.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e6559e9c0 con 0x557e6558d800 2024-09-13T23:43:27.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd7414a5700 1 -- 172.21.15.167:0/2864844210 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x557e65687080 con 0x557e6558d800 2024-09-13T23:43:27.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd749cfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd7414a5700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65928c00 0x557e65873600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd7414a5700 1 -- 172.21.15.167:0/2864844210 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x557e655f3340 con 0x557e6558d800 2024-09-13T23:43:27.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd747cb2700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65928c00 0x557e65873600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557e65929000 0x557e65873b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.543+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929400 con 0x557e65929000 2024-09-13T23:43:27.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd748cb4700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557e65929000 0x557e65873b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd747cb2700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65928c00 0x557e65873600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x557e6593dda0 tx=0x557e659e20c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd748cb4700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557e65929000 0x557e65873b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd749cfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd749cfaf40 20 realm 2024-09-13T23:43:27.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd749cfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929000 2024-09-13T23:43:27.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd749cfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd749cfaf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:27.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd749cfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929c00 con 0x557e65929000 2024-09-13T23:43:27.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd749cfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd749cfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557e65929400 0x557e65874100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929400 2024-09-13T23:43:27.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd7484b3700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557e65929400 0x557e65874100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.547+0000 7fd7484b3700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557e65929400 0x557e65874100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd749cfaf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:27.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd749cfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929c00 con 0x557e65929400 2024-09-13T23:43:27.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd749cfaf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:27.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd749cfaf40 20 searching for the correct realm 2024-09-13T23:43:27.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557e658ce000 0x557e65874680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658ce400 con 0x557e658ce000 2024-09-13T23:43:27.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd747cb2700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557e658ce000 0x557e65874680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd747cb2700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557e658ce000 0x557e65874680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658ce400 con 0x557e65929400 2024-09-13T23:43:27.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd7484b3700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557e658ce400 0x557e65874c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce400 2024-09-13T23:43:27.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd748cb4700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557e658ce400 0x557e65874c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.551+0000 7fd748cb4700 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557e658ce400 0x557e65874c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929000 2024-09-13T23:43:27.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929000 2024-09-13T23:43:27.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce000 2024-09-13T23:43:27.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929000 2024-09-13T23:43:27.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929400 2024-09-13T23:43:27.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929400 2024-09-13T23:43:27.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.555+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929000 2024-09-13T23:43:27.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce400 2024-09-13T23:43:27.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929000 2024-09-13T23:43:27.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce400 2024-09-13T23:43:27.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929400 2024-09-13T23:43:27.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce000 2024-09-13T23:43:27.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929000 2024-09-13T23:43:27.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.559+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929400 2024-09-13T23:43:27.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce000 2024-09-13T23:43:27.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929000 2024-09-13T23:43:27.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929000 2024-09-13T23:43:27.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929000 2024-09-13T23:43:27.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce000 2024-09-13T23:43:27.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce400 2024-09-13T23:43:27.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929000 2024-09-13T23:43:27.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.563+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce400 2024-09-13T23:43:27.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce000 2024-09-13T23:43:27.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce400 2024-09-13T23:43:27.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929400 2024-09-13T23:43:27.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929400 2024-09-13T23:43:27.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce000 2024-09-13T23:43:27.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929400 2024-09-13T23:43:27.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.567+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929400 2024-09-13T23:43:27.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:27.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:27.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:27.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:27.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658ce800 con 0x557e65929000 2024-09-13T23:43:27.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cec00 con 0x557e65929400 2024-09-13T23:43:27.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:27.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cf000 con 0x557e65929400 2024-09-13T23:43:27.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:27.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 zone default found 2024-09-13T23:43:27.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 4 Realm: () 2024-09-13T23:43:27.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:27.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:27.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:27.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 zonegroup default 2024-09-13T23:43:27.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cf400 con 0x557e658ce400 2024-09-13T23:43:27.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cf800 con 0x557e658ce000 2024-09-13T23:43:27.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.571+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.575+0000 7fd749cfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.575+0000 7fd749cfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.575+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cfc00 con 0x557e658ce000 2024-09-13T23:43:27.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.575+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.575+0000 7fd749cfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.575+0000 7fd749cfaf40 20 started sync module instance, tier type = 2024-09-13T23:43:27.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.575+0000 7fd749cfaf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:27.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.575+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cf800 con 0x557e65929400 2024-09-13T23:43:27.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.575+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.575+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94001357560576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cfc00 con 0x557e65929400 2024-09-13T23:43:27.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.575+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cec00 con 0x557e65929000 2024-09-13T23:43:27.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.579+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94001357560576] v33'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.579+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.579+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94001357561728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cf000 con 0x557e65929000 2024-09-13T23:43:27.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.579+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cf800 con 0x557e65929400 2024-09-13T23:43:27.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.583+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94001357561728] v33'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.583+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.583+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94001357562880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929c00 con 0x557e65929400 2024-09-13T23:43:27.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.583+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e658ce400 2024-09-13T23:43:27.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.587+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.587+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94001357564032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658ce800 con 0x557e658ce400 2024-09-13T23:43:27.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.587+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cf400 con 0x557e65929400 2024-09-13T23:43:27.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.587+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94001357562880] v33'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.587+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94001357564032] v33'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.587+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.587+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94001357565184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cec00 con 0x557e65929400 2024-09-13T23:43:27.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.587+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cf000 con 0x557e658ce000 2024-09-13T23:43:27.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.591+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94001357565184] v33'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.591+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.591+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94001357566336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658d0000 con 0x557e658ce000 2024-09-13T23:43:27.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.591+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658d0400 con 0x557e658ce000 2024-09-13T23:43:27.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.591+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94001357566336] v33'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94001357567488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658d0800 con 0x557e658ce000 2024-09-13T23:43:27.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658d0c00 con 0x557e658ce400 2024-09-13T23:43:27.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'48 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94001357568640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658d1000 con 0x557e658ce400 2024-09-13T23:43:27.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd749cfaf40 20 add_watcher() i=0 2024-09-13T23:43:27.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd749cfaf40 20 add_watcher() i=1 2024-09-13T23:43:27.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd749cfaf40 20 add_watcher() i=2 2024-09-13T23:43:27.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd749cfaf40 20 add_watcher() i=3 2024-09-13T23:43:27.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd749cfaf40 20 add_watcher() i=4 2024-09-13T23:43:27.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd749cfaf40 20 add_watcher() i=5 2024-09-13T23:43:27.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94001357567488] v33'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd749cfaf40 20 add_watcher() i=6 2024-09-13T23:43:27.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94001357568640] v33'49 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.595+0000 7fd749cfaf40 20 add_watcher() i=7 2024-09-13T23:43:27.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.599+0000 7fd749cfaf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:27.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.599+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658d1400 con 0x557e65929000 2024-09-13T23:43:27.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.599+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.599+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94001359233024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658d1800 con 0x557e65929000 2024-09-13T23:43:27.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.599+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94001359233024] v33'66 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.599+0000 7fd72f481700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:27.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.599+0000 7fd749cfaf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:27.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.599+0000 7fd749cfaf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:27.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.599+0000 7fd749cfaf40 1 Processor -- start 2024-09-13T23:43:27.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 -- start start 2024-09-13T23:43:27.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 0x557e65875180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e655b2780 con 0x557e658d0c00 2024-09-13T23:43:27.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd747cb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 0x557e65875180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd747cb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 0x557e65875180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50916/0 (socket says 172.21.15.167:50916) 2024-09-13T23:43:27.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd747cb2700 1 -- 172.21.15.167:0/1625969235 learned_addr learned my addr 172.21.15.167:0/1625969235 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd747cb2700 1 -- 172.21.15.167:0/1625969235 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6559eb60 con 0x557e658d0c00 2024-09-13T23:43:27.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd747cb2700 1 --2- 172.21.15.167:0/1625969235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 0x557e65875180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x557e65a0a090 tx=0x557e659e2060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e7409336e59236 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd72ec80700 1 -- 172.21.15.167:0/1625969235 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e65a12000 con 0x557e658d0c00 2024-09-13T23:43:27.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd72ec80700 1 -- 172.21.15.167:0/1625969235 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557e6593a600 con 0x557e658d0c00 2024-09-13T23:43:27.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd72ec80700 1 -- 172.21.15.167:0/1625969235 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e65a12160 con 0x557e658d0c00 2024-09-13T23:43:27.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 -- 172.21.15.167:0/1625969235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 msgr2=0x557e65875180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/1625969235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 0x557e65875180 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x557e65a0a090 tx=0x557e659e2060 comp rx=0 tx=0).stop 2024-09-13T23:43:27.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 -- 172.21.15.167:0/1625969235 shutdown_connections 2024-09-13T23:43:27.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/1625969235 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 0x557e65875180 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 -- 172.21.15.167:0/1625969235 >> 172.21.15.167:0/1625969235 conn(0x557e658d1c00 msgr2=0x557e6568e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 -- 172.21.15.167:0/1625969235 shutdown_connections 2024-09-13T23:43:27.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 -- 172.21.15.167:0/1625969235 wait complete. 2024-09-13T23:43:27.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 Processor -- start 2024-09-13T23:43:27.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 -- start start 2024-09-13T23:43:27.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 0x557e65875180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.603+0000 7fd749cfaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6561a000 con 0x557e658d0c00 2024-09-13T23:43:27.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd747cb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 0x557e65875180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd747cb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 0x557e65875180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50932/0 (socket says 172.21.15.167:50932) 2024-09-13T23:43:27.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd747cb2700 1 -- 172.21.15.167:0/242477083 learned_addr learned my addr 172.21.15.167:0/242477083 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd747cb2700 1 -- 172.21.15.167:0/242477083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6559ed00 con 0x557e658d0c00 2024-09-13T23:43:27.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd747cb2700 1 --2- 172.21.15.167:0/242477083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 0x557e65875180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x557e65a0b350 tx=0x557e659e2180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd72d47d700 1 -- 172.21.15.167:0/242477083 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e65a122c0 con 0x557e658d0c00 2024-09-13T23:43:27.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd72d47d700 1 -- 172.21.15.167:0/242477083 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557e6593a300 con 0x557e658d0c00 2024-09-13T23:43:27.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd749cfaf40 1 -- 172.21.15.167:0/242477083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557e6559e340 con 0x557e658d0c00 2024-09-13T23:43:27.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd72d47d700 1 -- 172.21.15.167:0/242477083 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557e65a12000 con 0x557e658d0c00 2024-09-13T23:43:27.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd749cfaf40 1 -- 172.21.15.167:0/242477083 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e6559eb60 con 0x557e658d0c00 2024-09-13T23:43:27.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd72d47d700 1 -- 172.21.15.167:0/242477083 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x557e65687b80 con 0x557e658d0c00 2024-09-13T23:43:27.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd72d47d700 1 --2- 172.21.15.167:0/242477083 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65a8c000 0x557e65a92000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd72d47d700 1 -- 172.21.15.167:0/242477083 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x557e655f3500 con 0x557e658d0c00 2024-09-13T23:43:27.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd748cb4700 1 --2- 172.21.15.167:0/242477083 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65a8c000 0x557e65a92000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.607+0000 7fd749cfaf40 5 note: GC not initialized 2024-09-13T23:43:27.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.611+0000 7fd748cb4700 1 --2- 172.21.15.167:0/242477083 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65a8c000 0x557e65a92000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x557e65991ad0 tx=0x557e658f1500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.611+0000 7fd72ac78700 20 reqs_thread_entry: start 2024-09-13T23:43:27.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.611+0000 7fd749cfaf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:27.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.611+0000 7fd749cfaf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:27.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.611+0000 7fd728473700 20 reqs_thread_entry: start 2024-09-13T23:43:27.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.611+0000 7fd749cfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.611+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658d0000 con 0x557e65929000 2024-09-13T23:43:27.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.611+0000 7fd728c74700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:57 2024 2024-09-13T23:43:27.620 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:27.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.611+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~459 out=459b,getxattrs out=75b] v0'0 uv4 ondisk = 0) v8 ==== 231+0+582 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.611+0000 7fd749cfaf40 20 rados_obj.operate() r=0 bl.length=459 2024-09-13T23:43:27.621 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:27.621 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:27.621 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:27.621 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-09-13T23:43:27.622 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:27.623 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:27.624 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:27.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.611+0000 7fd72f481700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:27.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.611+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94001359233024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658d0400 con 0x557e65929000 2024-09-13T23:43:27.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.615+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94001359233024] v33'67 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.615+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94001357560576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658ce800 con 0x557e65929400 2024-09-13T23:43:27.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.619+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94001357560576] v33'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.619+0000 7fd749cfaf40 20 remove_watcher() i=0 2024-09-13T23:43:27.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.619+0000 7fd749cfaf40 2 removed watcher, disabling cache 2024-09-13T23:43:27.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.619+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94001357561728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e658cf000 con 0x557e65929000 2024-09-13T23:43:27.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.619+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94001357561728] v33'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e65929000 2024-09-13T23:43:27.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.619+0000 7fd749cfaf40 20 remove_watcher() i=1 2024-09-13T23:43:27.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.619+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94001357562880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65929800 con 0x557e65929400 2024-09-13T23:43:27.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.623+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94001357562880] v33'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.623+0000 7fd749cfaf40 20 remove_watcher() i=2 2024-09-13T23:43:27.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.623+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94001357564032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65a8c400 con 0x557e658ce400 2024-09-13T23:43:27.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.623+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94001357564032] v33'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.623+0000 7fd749cfaf40 20 remove_watcher() i=3 2024-09-13T23:43:27.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.623+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94001357565184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65a8c800 con 0x557e65929400 2024-09-13T23:43:27.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.627+0000 7fd7484b3700 1 -- 172.21.15.167:0/2864844210 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94001357565184] v33'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5200 con 0x557e65929400 2024-09-13T23:43:27.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.627+0000 7fd749cfaf40 20 remove_watcher() i=4 2024-09-13T23:43:27.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.627+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94001357566336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65a8cc00 con 0x557e658ce000 2024-09-13T23:43:27.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.627+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94001357566336] v33'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.631+0000 7fd749cfaf40 20 remove_watcher() i=5 2024-09-13T23:43:27.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.631+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94001357567488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65a8d000 con 0x557e658ce000 2024-09-13T23:43:27.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.631+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94001357567488] v33'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e5440 con 0x557e658ce000 2024-09-13T23:43:27.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.631+0000 7fd749cfaf40 20 remove_watcher() i=6 2024-09-13T23:43:27.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.631+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94001357568640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557e65a8d400 con 0x557e658ce400 2024-09-13T23:43:27.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd748cb4700 1 -- 172.21.15.167:0/2864844210 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94001357568640] v33'50 uv48 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557e659e4b40 con 0x557e658ce400 2024-09-13T23:43:27.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 20 remove_watcher() i=7 2024-09-13T23:43:27.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557e65929400 msgr2=0x557e65874100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557e65929400 0x557e65874100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557e65929000 msgr2=0x557e65873b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557e65929000 0x557e65873b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557e658ce400 msgr2=0x557e65874c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557e658ce400 0x557e65874c00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557e658ce000 msgr2=0x557e65874680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557e658ce000 0x557e65874680 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65928c00 msgr2=0x557e65873600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65928c00 0x557e65873600 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x557e6593dda0 tx=0x557e659e20c0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 msgr2=0x557e65872b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 0x557e65872b00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x557e663f1950 tx=0x557e658f0a20 comp rx=0 tx=0).stop 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd747cb2700 1 -- 172.21.15.167:0/2864844210 reap_dead start 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 shutdown_connections 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/2864844210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d800 0x557e65872b00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 >> 172.21.15.167:0/2864844210 conn(0x557e6558d400 msgr2=0x557e6568d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 shutdown_connections 2024-09-13T23:43:27.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/2864844210 wait complete. 2024-09-13T23:43:27.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/242477083 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65a8c000 msgr2=0x557e65a92000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/242477083 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65a8c000 0x557e65a92000 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x557e65991ad0 tx=0x557e658f1500 comp rx=0 tx=0).stop 2024-09-13T23:43:27.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/242477083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 msgr2=0x557e65875180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/242477083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 0x557e65875180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x557e65a0b350 tx=0x557e659e2180 comp rx=0 tx=0).stop 2024-09-13T23:43:27.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/242477083 shutdown_connections 2024-09-13T23:43:27.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/242477083 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65a8c000 0x557e65a92000 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/242477083 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e658d0c00 0x557e65875180 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/242477083 >> 172.21.15.167:0/242477083 conn(0x557e658d1c00 msgr2=0x557e6568e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/242477083 shutdown_connections 2024-09-13T23:43:27.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/242477083 wait complete. 2024-09-13T23:43:27.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/3153849412 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65928000 msgr2=0x557e65873080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/3153849412 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65928000 0x557e65873080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x557e658b4f60 tx=0x557e658f0b40 comp rx=0 tx=0).stop 2024-09-13T23:43:27.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/3153849412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 msgr2=0x557e65872580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/3153849412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x557e663f16b0 tx=0x557e65691920 comp rx=0 tx=0).stop 2024-09-13T23:43:27.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/3153849412 shutdown_connections 2024-09-13T23:43:27.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/3153849412 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557e65928000 0x557e65873080 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 --2- 172.21.15.167:0/3153849412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557e6558d000 0x557e65872580 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/3153849412 >> 172.21.15.167:0/3153849412 conn(0x557e6558cc00 msgr2=0x557e6568d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/3153849412 shutdown_connections 2024-09-13T23:43:27.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.635+0000 7fd749cfaf40 1 -- 172.21.15.167:0/3153849412 wait complete. 2024-09-13T23:43:27.651 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:27.651 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-09-13T23:43:27.652 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-09-13T23:43:27.652 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --uid foo --access-key p5YnriCv1nAtykxBrupQ 2024-09-13T23:43:27.736 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:27.737 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:27.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.727+0000 7f27adc1df40 1 Processor -- start 2024-09-13T23:43:27.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.727+0000 7f27adc1df40 1 -- start start 2024-09-13T23:43:27.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.727+0000 7f27adc1df40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5634bf69d400 0x5634bf982580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.727+0000 7f27adc1df40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5634bf761500 con 0x5634bf69d000 2024-09-13T23:43:27.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.727+0000 7f27adc1df40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5634bf72aa00 con 0x5634bf69d400 2024-09-13T23:43:27.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.727+0000 7f27abbd5700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5634bf69d400 0x5634bf982580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.727+0000 7f27ac3d6700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5634bf69d000 0x5634bf79c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45290/0 (socket says 172.21.15.167:45290) 2024-09-13T23:43:27.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.727+0000 7f27ac3d6700 1 -- 172.21.15.167:0/2545228436 learned_addr learned my addr 172.21.15.167:0/2545228436 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:27.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27abbd5700 1 -- 172.21.15.167:0/2545228436 >> v1:172.21.15.12:6789/0 conn(0x5634bf69d000 legacy=0x5634bf79c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27abbd5700 1 -- 172.21.15.167:0/2545228436 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634bf6afd40 con 0x5634bf69d400 2024-09-13T23:43:27.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27abbd5700 1 --2- 172.21.15.167:0/2545228436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d400 0x5634bf982580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5634bf97d920 tx=0x5634bf7a1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1753bfcc99997f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27ab3d4700 1 -- 172.21.15.167:0/2545228436 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bf7414a0 con 0x5634bf69d400 2024-09-13T23:43:27.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27ab3d4700 1 -- 172.21.15.167:0/2545228436 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634bf761800 con 0x5634bf69d400 2024-09-13T23:43:27.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27ab3d4700 1 -- 172.21.15.167:0/2545228436 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bf741600 con 0x5634bf69d400 2024-09-13T23:43:27.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27adc1df40 1 -- 172.21.15.167:0/2545228436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d400 msgr2=0x5634bf982580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27adc1df40 1 --2- 172.21.15.167:0/2545228436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d400 0x5634bf982580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x5634bf97d920 tx=0x5634bf7a1920 comp rx=0 tx=0).stop 2024-09-13T23:43:27.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27adc1df40 1 -- 172.21.15.167:0/2545228436 shutdown_connections 2024-09-13T23:43:27.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27adc1df40 1 --2- 172.21.15.167:0/2545228436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d400 0x5634bf982580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27adc1df40 1 -- 172.21.15.167:0/2545228436 >> 172.21.15.167:0/2545228436 conn(0x5634bf69cc00 msgr2=0x5634bf79c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27adc1df40 1 -- 172.21.15.167:0/2545228436 shutdown_connections 2024-09-13T23:43:27.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27adc1df40 1 -- 172.21.15.167:0/2545228436 wait complete. 2024-09-13T23:43:27.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27adc1df40 1 Processor -- start 2024-09-13T23:43:27.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27adc1df40 1 -- start start 2024-09-13T23:43:27.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27adc1df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d000 0x5634bf982580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27adc1df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bf72a500 con 0x5634bf69d000 2024-09-13T23:43:27.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27ac3d6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d000 0x5634bf982580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27ac3d6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d000 0x5634bf982580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50948/0 (socket says 172.21.15.167:50948) 2024-09-13T23:43:27.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.731+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1505913239 learned_addr learned my addr 172.21.15.167:0/1505913239 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1505913239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634bf6afba0 con 0x5634bf69d000 2024-09-13T23:43:27.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27ac3d6700 1 --2- 172.21.15.167:0/1505913239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d000 0x5634bf982580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5634c05018c0 tx=0x5634bf7a1b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b295f3565ae05ee1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27aabd3700 1 -- 172.21.15.167:0/1505913239 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bf7414a0 con 0x5634bf69d000 2024-09-13T23:43:27.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27aabd3700 1 -- 172.21.15.167:0/1505913239 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634bf761b00 con 0x5634bf69d000 2024-09-13T23:43:27.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27aabd3700 1 -- 172.21.15.167:0/1505913239 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bf741600 con 0x5634bf69d000 2024-09-13T23:43:27.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27adc1df40 1 -- 172.21.15.167:0/1505913239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d000 msgr2=0x5634bf982580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1505913239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d000 0x5634bf982580 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x5634c05018c0 tx=0x5634bf7a1b60 comp rx=0 tx=0).stop 2024-09-13T23:43:27.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27adc1df40 1 -- 172.21.15.167:0/1505913239 shutdown_connections 2024-09-13T23:43:27.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1505913239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d000 0x5634bf982580 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27adc1df40 1 -- 172.21.15.167:0/1505913239 >> 172.21.15.167:0/1505913239 conn(0x5634bf69cc00 msgr2=0x5634bf79d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27adc1df40 1 -- 172.21.15.167:0/1505913239 shutdown_connections 2024-09-13T23:43:27.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27adc1df40 1 -- 172.21.15.167:0/1505913239 wait complete. 2024-09-13T23:43:27.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27adc1df40 1 Processor -- start 2024-09-13T23:43:27.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27adc1df40 1 -- start start 2024-09-13T23:43:27.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27adc1df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d400 0x5634bf982b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27adc1df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bf72a3c0 con 0x5634bf69d400 2024-09-13T23:43:27.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27ac3d6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d400 0x5634bf982b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27ac3d6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d400 0x5634bf982b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50964/0 (socket says 172.21.15.167:50964) 2024-09-13T23:43:27.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1757245595 learned_addr learned my addr 172.21.15.167:0/1757245595 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1757245595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634bf6afa00 con 0x5634bf69d400 2024-09-13T23:43:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27ac3d6700 1 --2- 172.21.15.167:0/1757245595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d400 0x5634bf982b00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5634c0501ad0 tx=0x5634bf7a1a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.735+0000 7f27a93d0700 1 -- 172.21.15.167:0/1757245595 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bf7418c0 con 0x5634bf69d400 2024-09-13T23:43:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27a93d0700 1 -- 172.21.15.167:0/1757245595 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634bf761c80 con 0x5634bf69d400 2024-09-13T23:43:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27adc1df40 1 -- 172.21.15.167:0/1757245595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5634bf6af860 con 0x5634bf69d400 2024-09-13T23:43:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27a93d0700 1 -- 172.21.15.167:0/1757245595 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bf7414a0 con 0x5634bf69d400 2024-09-13T23:43:27.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27adc1df40 1 -- 172.21.15.167:0/1757245595 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634bf6af6c0 con 0x5634bf69d400 2024-09-13T23:43:27.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27adc1df40 1 Processor -- start 2024-09-13T23:43:27.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27adc1df40 1 -- start start 2024-09-13T23:43:27.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27adc1df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 0x5634bf983080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27adc1df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bf6c2640 con 0x5634bf69dc00 2024-09-13T23:43:27.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27a93d0700 1 -- 172.21.15.167:0/1757245595 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5634bf796b00 con 0x5634bf69d400 2024-09-13T23:43:27.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27acbd7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 0x5634bf983080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27acbd7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 0x5634bf983080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50968/0 (socket says 172.21.15.167:50968) 2024-09-13T23:43:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27acbd7700 1 -- 172.21.15.167:0/2593626092 learned_addr learned my addr 172.21.15.167:0/2593626092 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27a93d0700 1 --2- 172.21.15.167:0/1757245595 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfa4c000 0x5634bf984100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27a93d0700 1 -- 172.21.15.167:0/1757245595 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x5634bf703180 con 0x5634bf69d400 2024-09-13T23:43:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27ac3d6700 1 --2- 172.21.15.167:0/1757245595 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfa4c000 0x5634bf984100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27acbd7700 1 -- 172.21.15.167:0/2593626092 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634bf6ae000 con 0x5634bf69dc00 2024-09-13T23:43:27.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27acbd7700 1 --2- 172.21.15.167:0/2593626092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 0x5634bf983080 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5634bfa24720 tx=0x5634bfa00de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=444484c1cbbeb10e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27ac3d6700 1 --2- 172.21.15.167:0/1757245595 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfa4c000 0x5634bf984100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5634c0515650 tx=0x5634bfa00240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27a6bcb700 1 -- 172.21.15.167:0/2593626092 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bf7418c0 con 0x5634bf69dc00 2024-09-13T23:43:27.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27a6bcb700 1 -- 172.21.15.167:0/2593626092 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634bfa52000 con 0x5634bf69dc00 2024-09-13T23:43:27.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.739+0000 7f27a6bcb700 1 -- 172.21.15.167:0/2593626092 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bf7414a0 con 0x5634bf69dc00 2024-09-13T23:43:27.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 -- 172.21.15.167:0/2593626092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 msgr2=0x5634bf983080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 --2- 172.21.15.167:0/2593626092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 0x5634bf983080 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x5634bfa24720 tx=0x5634bfa00de0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 -- 172.21.15.167:0/2593626092 shutdown_connections 2024-09-13T23:43:27.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 --2- 172.21.15.167:0/2593626092 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 0x5634bf983080 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 -- 172.21.15.167:0/2593626092 >> 172.21.15.167:0/2593626092 conn(0x5634bf69d800 msgr2=0x5634bf79d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 -- 172.21.15.167:0/2593626092 shutdown_connections 2024-09-13T23:43:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 -- 172.21.15.167:0/2593626092 wait complete. 2024-09-13T23:43:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 Processor -- start 2024-09-13T23:43:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 -- start start 2024-09-13T23:43:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 0x5634bf983080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bf6c28c0 con 0x5634bf69dc00 2024-09-13T23:43:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27acbd7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 0x5634bf983080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27acbd7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 0x5634bf983080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50970/0 (socket says 172.21.15.167:50970) 2024-09-13T23:43:27.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 learned_addr learned my addr 172.21.15.167:0/1213382077 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634bf6aeea0 con 0x5634bf69dc00 2024-09-13T23:43:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27acbd7700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 0x5634bf983080 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x5634c0514900 tx=0x5634bfa00840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27a53c8700 1 -- 172.21.15.167:0/1213382077 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bf741b80 con 0x5634bf69dc00 2024-09-13T23:43:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27a53c8700 1 -- 172.21.15.167:0/1213382077 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634bfa52180 con 0x5634bf69dc00 2024-09-13T23:43:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5634bf6ae820 con 0x5634bf69dc00 2024-09-13T23:43:27.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27a53c8700 1 -- 172.21.15.167:0/1213382077 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bf7418c0 con 0x5634bf69dc00 2024-09-13T23:43:27.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.743+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634bf6ae9c0 con 0x5634bf69dc00 2024-09-13T23:43:27.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27a53c8700 1 -- 172.21.15.167:0/1213382077 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5634bf796dc0 con 0x5634bf69dc00 2024-09-13T23:43:27.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27a53c8700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfa4cc00 0x5634bf984680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27a53c8700 1 -- 172.21.15.167:0/1213382077 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x5634bf703340 con 0x5634bf69dc00 2024-09-13T23:43:27.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27abbd5700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfa4cc00 0x5634bf984680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5634bfa4d000 0x5634bf983600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfa4d400 con 0x5634bfa4d000 2024-09-13T23:43:27.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27acbd7700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5634bfa4d000 0x5634bf983600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27abbd5700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfa4cc00 0x5634bf984680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5634bf97d920 tx=0x5634bf7a1800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27acbd7700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5634bfa4d000 0x5634bf983600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 20 realm 2024-09-13T23:43:27.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfa4d800 con 0x5634bfa4d000 2024-09-13T23:43:27.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:27.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfa4dc00 con 0x5634bfa4d000 2024-09-13T23:43:27.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5634bfa4d400 0x5634bf983b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.747+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfa4d800 con 0x5634bfa4d400 2024-09-13T23:43:27.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27ac3d6700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5634bfa4d400 0x5634bf983b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27ac3d6700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5634bfa4d400 0x5634bf983b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27adc1df40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:27.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27adc1df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfa4dc00 con 0x5634bfa4d400 2024-09-13T23:43:27.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27adc1df40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:27.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27adc1df40 20 searching for the correct realm 2024-09-13T23:43:27.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5634bfa4d800 0x5634bf984c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfa4dc00 con 0x5634bfa4d800 2024-09-13T23:43:27.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27abbd5700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5634bfa4d800 0x5634bf984c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27abbd5700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5634bfa4d800 0x5634bf984c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.751+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfa4dc00 con 0x5634bfa4d400 2024-09-13T23:43:27.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27ac3d6700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5634bfa4dc00 0x5634bf982580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4dc00 2024-09-13T23:43:27.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27acbd7700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5634bfa4dc00 0x5634bf982580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27acbd7700 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5634bfa4dc00 0x5634bf982580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d000 2024-09-13T23:43:27.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d000 2024-09-13T23:43:27.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d800 2024-09-13T23:43:27.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d000 2024-09-13T23:43:27.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.755+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d400 2024-09-13T23:43:27.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.759+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.759+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d400 2024-09-13T23:43:27.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.759+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.759+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d000 2024-09-13T23:43:27.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.759+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.759+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4dc00 2024-09-13T23:43:27.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.759+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.759+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d000 2024-09-13T23:43:27.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.759+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.759+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4dc00 2024-09-13T23:43:27.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.759+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.759+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d400 2024-09-13T23:43:27.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d800 2024-09-13T23:43:27.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d000 2024-09-13T23:43:27.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d400 2024-09-13T23:43:27.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d800 2024-09-13T23:43:27.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d000 2024-09-13T23:43:27.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d000 2024-09-13T23:43:27.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.763+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d000 2024-09-13T23:43:27.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d800 2024-09-13T23:43:27.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4dc00 2024-09-13T23:43:27.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d000 2024-09-13T23:43:27.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4dc00 2024-09-13T23:43:27.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d800 2024-09-13T23:43:27.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4dc00 2024-09-13T23:43:27.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.767+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d400 2024-09-13T23:43:27.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d400 2024-09-13T23:43:27.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d800 2024-09-13T23:43:27.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d400 2024-09-13T23:43:27.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d400 2024-09-13T23:43:27.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27adc1df40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:27.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27adc1df40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:27.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27adc1df40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:27.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27adc1df40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:27.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27adc1df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.771+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c800 con 0x5634bfa4d000 2024-09-13T23:43:27.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0cc00 con 0x5634bfa4d400 2024-09-13T23:43:27.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:27.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0d000 con 0x5634bfa4d400 2024-09-13T23:43:27.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:27.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 zone default found 2024-09-13T23:43:27.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 4 Realm: () 2024-09-13T23:43:27.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:27.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:27.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:27.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 zonegroup default 2024-09-13T23:43:27.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0d400 con 0x5634bfa4dc00 2024-09-13T23:43:27.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0d800 con 0x5634bfa4d800 2024-09-13T23:43:27.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0dc00 con 0x5634bfa4d800 2024-09-13T23:43:27.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 started sync module instance, tier type = 2024-09-13T23:43:27.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:27.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.775+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0d800 con 0x5634bfa4d400 2024-09-13T23:43:27.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.779+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.779+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94784552672000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0dc00 con 0x5634bfa4d400 2024-09-13T23:43:27.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.779+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c400 con 0x5634bfa4d000 2024-09-13T23:43:27.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.783+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.783+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94784552672000] v33'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.783+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94784552673152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d000 2024-09-13T23:43:27.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.783+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0cc00 con 0x5634bfa4d400 2024-09-13T23:43:27.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.787+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94784552673152] v33'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.787+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.787+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94784552674304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0d000 con 0x5634bfa4d400 2024-09-13T23:43:27.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.787+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c800 con 0x5634bfa4dc00 2024-09-13T23:43:27.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.787+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.787+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94784552675456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0d400 con 0x5634bfa4dc00 2024-09-13T23:43:27.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.787+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c400 con 0x5634bfa4d400 2024-09-13T23:43:27.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.791+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94784552674304] v33'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.791+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94784552675456] v33'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.791+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.791+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94784552676608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d400 2024-09-13T23:43:27.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.791+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0d800 con 0x5634bfa4d800 2024-09-13T23:43:27.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.795+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.795+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94784552677760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0dc00 con 0x5634bfa4d800 2024-09-13T23:43:27.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.795+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0cc00 con 0x5634bfa4d800 2024-09-13T23:43:27.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.795+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94784552676608] v33'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.795+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.795+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94784552678912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0d000 con 0x5634bfa4d800 2024-09-13T23:43:27.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.795+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c400 con 0x5634bfa4dc00 2024-09-13T23:43:27.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.795+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94784552677760] v33'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'51 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94784552680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0d800 con 0x5634bfa4dc00 2024-09-13T23:43:27.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27adc1df40 20 add_watcher() i=0 2024-09-13T23:43:27.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94784552678912] v33'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27adc1df40 20 add_watcher() i=1 2024-09-13T23:43:27.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27adc1df40 20 add_watcher() i=2 2024-09-13T23:43:27.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27adc1df40 20 add_watcher() i=3 2024-09-13T23:43:27.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27adc1df40 20 add_watcher() i=4 2024-09-13T23:43:27.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27adc1df40 20 add_watcher() i=5 2024-09-13T23:43:27.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27adc1df40 20 add_watcher() i=6 2024-09-13T23:43:27.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94784552680064] v33'52 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27adc1df40 20 add_watcher() i=7 2024-09-13T23:43:27.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27adc1df40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:27.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0dc00 con 0x5634bfa4d000 2024-09-13T23:43:27.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.799+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94784554369024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0cc00 con 0x5634bfa4d000 2024-09-13T23:43:27.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.803+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94784554369024] v33'68 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.803+0000 7f27933a4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:27.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.803+0000 7f27adc1df40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:27.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.803+0000 7f27adc1df40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:27.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.803+0000 7f27adc1df40 1 Processor -- start 2024-09-13T23:43:27.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.803+0000 7f27adc1df40 1 -- start start 2024-09-13T23:43:27.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.803+0000 7f27adc1df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 0x5634bf985180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.803+0000 7f27adc1df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bf72a3c0 con 0x5634bfb0d400 2024-09-13T23:43:27.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.803+0000 7f27abbd5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 0x5634bf985180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.803+0000 7f27abbd5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 0x5634bf985180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50982/0 (socket says 172.21.15.167:50982) 2024-09-13T23:43:27.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.803+0000 7f27abbd5700 1 -- 172.21.15.167:0/986203965 learned_addr learned my addr 172.21.15.167:0/986203965 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27abbd5700 1 -- 172.21.15.167:0/986203965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634bf6aeb60 con 0x5634bfb0d400 2024-09-13T23:43:27.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27abbd5700 1 --2- 172.21.15.167:0/986203965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 0x5634bf985180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5634bfab5fb0 tx=0x5634bf7a1860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9e9645a2bea6d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f2792ba3700 1 -- 172.21.15.167:0/986203965 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bf741760 con 0x5634bfb0d400 2024-09-13T23:43:27.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f2792ba3700 1 -- 172.21.15.167:0/986203965 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634bfa52600 con 0x5634bfb0d400 2024-09-13T23:43:27.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f2792ba3700 1 -- 172.21.15.167:0/986203965 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bfb2c000 con 0x5634bfb0d400 2024-09-13T23:43:27.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27adc1df40 1 -- 172.21.15.167:0/986203965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 msgr2=0x5634bf985180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27adc1df40 1 --2- 172.21.15.167:0/986203965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 0x5634bf985180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5634bfab5fb0 tx=0x5634bf7a1860 comp rx=0 tx=0).stop 2024-09-13T23:43:27.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27adc1df40 1 -- 172.21.15.167:0/986203965 shutdown_connections 2024-09-13T23:43:27.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27adc1df40 1 --2- 172.21.15.167:0/986203965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 0x5634bf985180 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27adc1df40 1 -- 172.21.15.167:0/986203965 >> 172.21.15.167:0/986203965 conn(0x5634bfb0c800 msgr2=0x5634bf79e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27adc1df40 1 -- 172.21.15.167:0/986203965 shutdown_connections 2024-09-13T23:43:27.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27adc1df40 1 -- 172.21.15.167:0/986203965 wait complete. 2024-09-13T23:43:27.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27adc1df40 1 Processor -- start 2024-09-13T23:43:27.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27adc1df40 1 -- start start 2024-09-13T23:43:27.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27adc1df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 0x5634bf985180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27adc1df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bf72a500 con 0x5634bfb0d400 2024-09-13T23:43:27.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27abbd5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 0x5634bf985180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27abbd5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 0x5634bf985180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:50990/0 (socket says 172.21.15.167:50990) 2024-09-13T23:43:27.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27abbd5700 1 -- 172.21.15.167:0/2555961542 learned_addr learned my addr 172.21.15.167:0/2555961542 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.807+0000 7f27abbd5700 1 -- 172.21.15.167:0/2555961542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634bf6aed00 con 0x5634bfb0d400 2024-09-13T23:43:27.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.811+0000 7f27abbd5700 1 --2- 172.21.15.167:0/2555961542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 0x5634bf985180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5634bfb23290 tx=0x5634bfb24300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.811+0000 7f27913a0700 1 -- 172.21.15.167:0/2555961542 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bfb2c160 con 0x5634bfb0d400 2024-09-13T23:43:27.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.811+0000 7f27913a0700 1 -- 172.21.15.167:0/2555961542 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5634bfa52300 con 0x5634bfb0d400 2024-09-13T23:43:27.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.811+0000 7f27adc1df40 1 -- 172.21.15.167:0/2555961542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5634bf6ae340 con 0x5634bfb0d400 2024-09-13T23:43:27.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.811+0000 7f27913a0700 1 -- 172.21.15.167:0/2555961542 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5634bfb2c2c0 con 0x5634bfb0d400 2024-09-13T23:43:27.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.811+0000 7f27adc1df40 1 -- 172.21.15.167:0/2555961542 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634bf6afd40 con 0x5634bfb0d400 2024-09-13T23:43:27.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.811+0000 7f27913a0700 1 -- 172.21.15.167:0/2555961542 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5634bf797b80 con 0x5634bfb0d400 2024-09-13T23:43:27.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.811+0000 7f27913a0700 1 --2- 172.21.15.167:0/2555961542 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfb9e000 0x5634bfba4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.811+0000 7f27913a0700 1 -- 172.21.15.167:0/2555961542 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x5634bf703500 con 0x5634bfb0d400 2024-09-13T23:43:27.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.811+0000 7f27acbd7700 1 --2- 172.21.15.167:0/2555961542 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfb9e000 0x5634bfba4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.811+0000 7f27adc1df40 5 note: GC not initialized 2024-09-13T23:43:27.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.811+0000 7f27acbd7700 1 --2- 172.21.15.167:0/2555961542 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfb9e000 0x5634bfba4000 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5634bfab5560 tx=0x5634bfa017a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f27adc1df40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:27.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f278eb9b700 20 reqs_thread_entry: start 2024-09-13T23:43:27.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f27adc1df40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:27.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f278cb97700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:58 2024 2024-09-13T23:43:27.824 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:27.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f278c375700 20 reqs_thread_entry: start 2024-09-13T23:43:27.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f27adc1df40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:27.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f27adc1df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0dc00 con 0x5634bfa4d000 2024-09-13T23:43:27.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~459 out=459b,getxattrs out=75b] v0'0 uv4 ondisk = 0) v8 ==== 231+0+582 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f27adc1df40 20 rados_obj.operate() r=0 bl.length=459 2024-09-13T23:43:27.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f27adc1df40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:27.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f27adc1df40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:27.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f27adc1df40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:27.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.815+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb9e800 con 0x5634bfa4d000 2024-09-13T23:43:27.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~357] v33'5 uv5 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27adc1df40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:27.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27adc1df40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:27.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27adc1df40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:27.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94784554371328 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb9ec00 con 0x5634bfa4d400 2024-09-13T23:43:27.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94784552676608 notify 141733920773 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x5634bf9fa820 con 0x5634bfa4d400 2024-09-13T23:43:27.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27a6bcb700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920773 cookie 94784552676608 notifier 4366 bl.length()=533 2024-09-13T23:43:27.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27a6bcb700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:27.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27a6bcb700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:27.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(64 notify.4 [notify cookie 94784554371328 out=8b] v0'0 uv51 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27a6bcb700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfc10000 con 0x5634bfa4d400 2024-09-13T23:43:27.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94784554371328 notify 141733920773 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5634bf9fa9c0 con 0x5634bfa4d400 2024-09-13T23:43:27.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.819+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:66 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb9f000 con 0x5634bfa4d800 2024-09-13T23:43:27.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.823+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(66 p5YnriCv1nAtykxBrupQ [delete] v33'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.823+0000 7f27adc1df40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-13T23:43:27.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.823+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94784554371328 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb9f400 con 0x5634bfa4d400 2024-09-13T23:43:27.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.823+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== watch-notify(notify (1) cookie 94784552674304 notify 141733920774 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x5634bf9fa9c0 con 0x5634bfa4d400 2024-09-13T23:43:27.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.823+0000 7f27a6bcb700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920774 cookie 94784552674304 notifier 4366 bl.length()=170 2024-09-13T23:43:27.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.823+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(67 notify.2 [notify cookie 94784554371328 out=8b] v0'0 uv51 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.823+0000 7f27a6bcb700 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfc10400 con 0x5634bfa4d400 2024-09-13T23:43:27.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.823+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.823+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== watch-notify(notify_complete (2) cookie 94784554371328 notify 141733920774 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5634bf9fab60 con 0x5634bfa4d400 2024-09-13T23:43:27.833 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:27.833 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:27.833 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:27.833 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:27.833 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:27.833 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:27.833 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:27.833 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:27.833 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:27.834 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:27.835 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:27.836 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:27.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.823+0000 7f27933a4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:27.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.823+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94784554369024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb9f800 con 0x5634bfa4d000 2024-09-13T23:43:27.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.827+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94784554369024] v33'69 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.827+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94784552672000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb9fc00 con 0x5634bfa4d400 2024-09-13T23:43:27.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.831+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94784552672000] v33'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.831+0000 7f27adc1df40 20 remove_watcher() i=0 2024-09-13T23:43:27.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.831+0000 7f27adc1df40 2 removed watcher, disabling cache 2024-09-13T23:43:27.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.831+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94784552673152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb9ec00 con 0x5634bfa4d000 2024-09-13T23:43:27.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.831+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94784552673152] v33'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4d000 2024-09-13T23:43:27.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.831+0000 7f27adc1df40 20 remove_watcher() i=1 2024-09-13T23:43:27.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.831+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94784552674304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb9f400 con 0x5634bfa4d400 2024-09-13T23:43:27.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.835+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94784552674304] v33'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.835+0000 7f27adc1df40 20 remove_watcher() i=2 2024-09-13T23:43:27.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.835+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94784552675456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb9fc00 con 0x5634bfa4dc00 2024-09-13T23:43:27.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.839+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94784552675456] v33'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.839+0000 7f27adc1df40 20 remove_watcher() i=3 2024-09-13T23:43:27.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.839+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94784552676608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb0c000 con 0x5634bfa4d400 2024-09-13T23:43:27.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.839+0000 7f27ac3d6700 1 -- 172.21.15.167:0/1213382077 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94784552676608] v33'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3200 con 0x5634bfa4d400 2024-09-13T23:43:27.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.839+0000 7f27adc1df40 20 remove_watcher() i=4 2024-09-13T23:43:27.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.839+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94784552677760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb9e800 con 0x5634bfa4d800 2024-09-13T23:43:27.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.843+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94784552677760] v33'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.843+0000 7f27adc1df40 20 remove_watcher() i=5 2024-09-13T23:43:27.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.843+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94784552678912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb9e400 con 0x5634bfa4d800 2024-09-13T23:43:27.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.843+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94784552678912] v33'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad3440 con 0x5634bfa4d800 2024-09-13T23:43:27.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.843+0000 7f27adc1df40 20 remove_watcher() i=6 2024-09-13T23:43:27.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.843+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94784552680064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5634bfb9f800 con 0x5634bfa4dc00 2024-09-13T23:43:27.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27acbd7700 1 -- 172.21.15.167:0/1213382077 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94784552680064] v33'53 uv51 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5634bfad2b40 con 0x5634bfa4dc00 2024-09-13T23:43:27.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 20 remove_watcher() i=7 2024-09-13T23:43:27.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5634bfa4d400 msgr2=0x5634bf983b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5634bfa4d400 0x5634bf983b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5634bfa4d000 msgr2=0x5634bf983600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5634bfa4d000 0x5634bf983600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5634bfa4dc00 msgr2=0x5634bf982580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5634bfa4dc00 0x5634bf982580 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5634bfa4d800 msgr2=0x5634bf984c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5634bfa4d800 0x5634bf984c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfa4cc00 msgr2=0x5634bf984680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfa4cc00 0x5634bf984680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x5634bf97d920 tx=0x5634bf7a1800 comp rx=0 tx=0).stop 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 msgr2=0x5634bf983080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27abbd5700 1 -- 172.21.15.167:0/1213382077 reap_dead start 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 0x5634bf983080 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x5634c0514900 tx=0x5634bfa00840 comp rx=0 tx=0).stop 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 shutdown_connections 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1213382077 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69dc00 0x5634bf983080 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 >> 172.21.15.167:0/1213382077 conn(0x5634bf69d800 msgr2=0x5634bf79d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 shutdown_connections 2024-09-13T23:43:27.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 -- 172.21.15.167:0/1213382077 wait complete. 2024-09-13T23:43:27.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 -- 172.21.15.167:0/2555961542 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfb9e000 msgr2=0x5634bfba4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 --2- 172.21.15.167:0/2555961542 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfb9e000 0x5634bfba4000 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x5634bfab5560 tx=0x5634bfa017a0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 -- 172.21.15.167:0/2555961542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 msgr2=0x5634bf985180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.847+0000 7f27adc1df40 1 --2- 172.21.15.167:0/2555961542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 0x5634bf985180 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x5634bfb23290 tx=0x5634bfb24300 comp rx=0 tx=0).stop 2024-09-13T23:43:27.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 -- 172.21.15.167:0/2555961542 shutdown_connections 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 --2- 172.21.15.167:0/2555961542 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfb9e000 0x5634bfba4000 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 --2- 172.21.15.167:0/2555961542 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bfb0d400 0x5634bf985180 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 -- 172.21.15.167:0/2555961542 >> 172.21.15.167:0/2555961542 conn(0x5634bfb0c800 msgr2=0x5634bf79e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 -- 172.21.15.167:0/2555961542 shutdown_connections 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 -- 172.21.15.167:0/2555961542 wait complete. 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 -- 172.21.15.167:0/1757245595 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfa4c000 msgr2=0x5634bf984100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1757245595 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfa4c000 0x5634bf984100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x5634c0515650 tx=0x5634bfa00240 comp rx=0 tx=0).stop 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 -- 172.21.15.167:0/1757245595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d400 msgr2=0x5634bf982b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1757245595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d400 0x5634bf982b00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5634c0501ad0 tx=0x5634bf7a1a40 comp rx=0 tx=0).stop 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 -- 172.21.15.167:0/1757245595 shutdown_connections 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1757245595 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5634bfa4c000 0x5634bf984100 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 --2- 172.21.15.167:0/1757245595 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5634bf69d400 0x5634bf982b00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 -- 172.21.15.167:0/1757245595 >> 172.21.15.167:0/1757245595 conn(0x5634bf69cc00 msgr2=0x5634bf79d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 -- 172.21.15.167:0/1757245595 shutdown_connections 2024-09-13T23:43:27.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.851+0000 7f27adc1df40 1 -- 172.21.15.167:0/1757245595 wait complete. 2024-09-13T23:43:27.864 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:27.864 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-09-13T23:43:27.864 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-09-13T23:43:27.864 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo1 --access full 2024-09-13T23:43:27.908 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:27.908 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:27.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.899+0000 7f99c190cf40 1 Processor -- start 2024-09-13T23:43:27.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.899+0000 7f99c190cf40 1 -- start start 2024-09-13T23:43:27.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.899+0000 7f99c190cf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55cd9d4bd400 0x55cd9d7a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.899+0000 7f99c190cf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55cd9d581500 con 0x55cd9d4bd000 2024-09-13T23:43:27.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.899+0000 7f99c190cf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55cd9d54aa00 con 0x55cd9d4bd400 2024-09-13T23:43:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.899+0000 7f99bf8c4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55cd9d4bd400 0x55cd9d7a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.899+0000 7f99c00c5700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55cd9d4bd000 0x55cd9d5bc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45296/0 (socket says 172.21.15.167:45296) 2024-09-13T23:43:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.899+0000 7f99bf8c4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55cd9d4bd400 0x55cd9d7a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51004/0 (socket says 172.21.15.167:51004) 2024-09-13T23:43:27.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.899+0000 7f99c00c5700 1 -- 172.21.15.167:0/4126896817 learned_addr learned my addr 172.21.15.167:0/4126896817 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:27.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99bf8c4700 1 -- 172.21.15.167:0/4126896817 >> v1:172.21.15.12:6789/0 conn(0x55cd9d4bd000 legacy=0x55cd9d5bc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99bf8c4700 1 -- 172.21.15.167:0/4126896817 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9d4cfd40 con 0x55cd9d4bd400 2024-09-13T23:43:27.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99bf8c4700 1 --2- 172.21.15.167:0/4126896817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd400 0x55cd9d7a2580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55cd9d79d920 tx=0x55cd9d5c1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7615e38a55c566e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99bf0c3700 1 -- 172.21.15.167:0/4126896817 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d5614a0 con 0x55cd9d4bd400 2024-09-13T23:43:27.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99bf0c3700 1 -- 172.21.15.167:0/4126896817 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd9d581800 con 0x55cd9d4bd400 2024-09-13T23:43:27.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99bf0c3700 1 -- 172.21.15.167:0/4126896817 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d561600 con 0x55cd9d4bd400 2024-09-13T23:43:27.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c190cf40 1 -- 172.21.15.167:0/4126896817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd400 msgr2=0x55cd9d7a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c190cf40 1 --2- 172.21.15.167:0/4126896817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd400 0x55cd9d7a2580 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55cd9d79d920 tx=0x55cd9d5c1920 comp rx=0 tx=0).stop 2024-09-13T23:43:27.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c190cf40 1 -- 172.21.15.167:0/4126896817 shutdown_connections 2024-09-13T23:43:27.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c190cf40 1 --2- 172.21.15.167:0/4126896817 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd400 0x55cd9d7a2580 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c190cf40 1 -- 172.21.15.167:0/4126896817 >> 172.21.15.167:0/4126896817 conn(0x55cd9d4bcc00 msgr2=0x55cd9d5bc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c190cf40 1 -- 172.21.15.167:0/4126896817 shutdown_connections 2024-09-13T23:43:27.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c190cf40 1 -- 172.21.15.167:0/4126896817 wait complete. 2024-09-13T23:43:27.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c190cf40 1 Processor -- start 2024-09-13T23:43:27.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c190cf40 1 -- start start 2024-09-13T23:43:27.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c190cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 0x55cd9d7a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c190cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9d54a500 con 0x55cd9d4bd000 2024-09-13T23:43:27.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c00c5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 0x55cd9d7a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c00c5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 0x55cd9d7a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51014/0 (socket says 172.21.15.167:51014) 2024-09-13T23:43:27.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c00c5700 1 -- 172.21.15.167:0/3834771914 learned_addr learned my addr 172.21.15.167:0/3834771914 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.903+0000 7f99c00c5700 1 -- 172.21.15.167:0/3834771914 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9d4cfba0 con 0x55cd9d4bd000 2024-09-13T23:43:27.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c00c5700 1 --2- 172.21.15.167:0/3834771914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 0x55cd9d7a2580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55cd9d619e60 tx=0x55cd9d5c1b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d1cfcbc1fe8db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99be8c2700 1 -- 172.21.15.167:0/3834771914 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d5614a0 con 0x55cd9d4bd000 2024-09-13T23:43:27.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99be8c2700 1 -- 172.21.15.167:0/3834771914 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd9d581b00 con 0x55cd9d4bd000 2024-09-13T23:43:27.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99be8c2700 1 -- 172.21.15.167:0/3834771914 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d561600 con 0x55cd9d4bd000 2024-09-13T23:43:27.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 -- 172.21.15.167:0/3834771914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 msgr2=0x55cd9d7a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 --2- 172.21.15.167:0/3834771914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 0x55cd9d7a2580 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55cd9d619e60 tx=0x55cd9d5c1b60 comp rx=0 tx=0).stop 2024-09-13T23:43:27.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 -- 172.21.15.167:0/3834771914 shutdown_connections 2024-09-13T23:43:27.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 --2- 172.21.15.167:0/3834771914 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 0x55cd9d7a2580 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 -- 172.21.15.167:0/3834771914 >> 172.21.15.167:0/3834771914 conn(0x55cd9d4bcc00 msgr2=0x55cd9d5bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 -- 172.21.15.167:0/3834771914 shutdown_connections 2024-09-13T23:43:27.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 -- 172.21.15.167:0/3834771914 wait complete. 2024-09-13T23:43:27.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 Processor -- start 2024-09-13T23:43:27.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 -- start start 2024-09-13T23:43:27.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 0x55cd9d7a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9d54a3c0 con 0x55cd9d4bd000 2024-09-13T23:43:27.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c00c5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 0x55cd9d7a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c00c5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 0x55cd9d7a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51024/0 (socket says 172.21.15.167:51024) 2024-09-13T23:43:27.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c00c5700 1 -- 172.21.15.167:0/3387852329 learned_addr learned my addr 172.21.15.167:0/3387852329 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c00c5700 1 -- 172.21.15.167:0/3387852329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9d4cfa00 con 0x55cd9d4bd000 2024-09-13T23:43:27.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c00c5700 1 --2- 172.21.15.167:0/3387852329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 0x55cd9d7a2580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55cd9e32e0c0 tx=0x55cd9d5c1a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99bd0bf700 1 -- 172.21.15.167:0/3387852329 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d5618c0 con 0x55cd9d4bd000 2024-09-13T23:43:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99bd0bf700 1 -- 172.21.15.167:0/3387852329 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd9d581c80 con 0x55cd9d4bd000 2024-09-13T23:43:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 -- 172.21.15.167:0/3387852329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd9d4cf860 con 0x55cd9d4bd000 2024-09-13T23:43:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99bd0bf700 1 -- 172.21.15.167:0/3387852329 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d5614a0 con 0x55cd9d4bd000 2024-09-13T23:43:27.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.907+0000 7f99c190cf40 1 -- 172.21.15.167:0/3387852329 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd9d4cf6c0 con 0x55cd9d4bd000 2024-09-13T23:43:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99c190cf40 1 Processor -- start 2024-09-13T23:43:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99bd0bf700 1 -- 172.21.15.167:0/3387852329 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55cd9d5b6b00 con 0x55cd9d4bd000 2024-09-13T23:43:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99c190cf40 1 -- start start 2024-09-13T23:43:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99bd0bf700 1 --2- 172.21.15.167:0/3387852329 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d860000 0x55cd9d7a3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99c190cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 0x55cd9d7a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99bd0bf700 1 -- 172.21.15.167:0/3387852329 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55cd9d523180 con 0x55cd9d4bd000 2024-09-13T23:43:27.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99c190cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9d4e2640 con 0x55cd9d4bd800 2024-09-13T23:43:27.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99c08c6700 1 --2- 172.21.15.167:0/3387852329 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d860000 0x55cd9d7a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99c00c5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 0x55cd9d7a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99c00c5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 0x55cd9d7a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51038/0 (socket says 172.21.15.167:51038) 2024-09-13T23:43:27.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99c00c5700 1 -- 172.21.15.167:0/2773647298 learned_addr learned my addr 172.21.15.167:0/2773647298 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99c08c6700 1 --2- 172.21.15.167:0/3387852329 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d860000 0x55cd9d7a3080 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55cd9d619f80 tx=0x55cd9d81cde0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99c00c5700 1 -- 172.21.15.167:0/2773647298 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9d4ce000 con 0x55cd9d4bd800 2024-09-13T23:43:27.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99c00c5700 1 --2- 172.21.15.167:0/2773647298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 0x55cd9d7a2b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55cd9e32f080 tx=0x55cd9d81c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa5ac12faf2da12 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99ba8ba700 1 -- 172.21.15.167:0/2773647298 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d561600 con 0x55cd9d4bd800 2024-09-13T23:43:27.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99ba8ba700 1 -- 172.21.15.167:0/2773647298 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd9d581c80 con 0x55cd9d4bd800 2024-09-13T23:43:27.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.911+0000 7f99ba8ba700 1 -- 172.21.15.167:0/2773647298 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d561a20 con 0x55cd9d4bd800 2024-09-13T23:43:27.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c190cf40 1 -- 172.21.15.167:0/2773647298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 msgr2=0x55cd9d7a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2773647298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 0x55cd9d7a2b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55cd9e32f080 tx=0x55cd9d81c240 comp rx=0 tx=0).stop 2024-09-13T23:43:27.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c190cf40 1 -- 172.21.15.167:0/2773647298 shutdown_connections 2024-09-13T23:43:27.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2773647298 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 0x55cd9d7a2b00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c190cf40 1 -- 172.21.15.167:0/2773647298 >> 172.21.15.167:0/2773647298 conn(0x55cd9d4bd400 msgr2=0x55cd9d5bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c190cf40 1 -- 172.21.15.167:0/2773647298 shutdown_connections 2024-09-13T23:43:27.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c190cf40 1 -- 172.21.15.167:0/2773647298 wait complete. 2024-09-13T23:43:27.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c190cf40 1 Processor -- start 2024-09-13T23:43:27.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c190cf40 1 -- start start 2024-09-13T23:43:27.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c190cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 0x55cd9d7a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c190cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9d4e28c0 con 0x55cd9d4bd800 2024-09-13T23:43:27.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c00c5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 0x55cd9d7a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c00c5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 0x55cd9d7a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51044/0 (socket says 172.21.15.167:51044) 2024-09-13T23:43:27.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 learned_addr learned my addr 172.21.15.167:0/2037993769 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9d4ceea0 con 0x55cd9d4bd800 2024-09-13T23:43:27.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.915+0000 7f99c00c5700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 0x55cd9d7a2b00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55cd9e32e2a0 tx=0x55cd9d81cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99b90b7700 1 -- 172.21.15.167:0/2037993769 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d561600 con 0x55cd9d4bd800 2024-09-13T23:43:27.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99b90b7700 1 -- 172.21.15.167:0/2037993769 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd9d880000 con 0x55cd9d4bd800 2024-09-13T23:43:27.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99b90b7700 1 -- 172.21.15.167:0/2037993769 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d561a20 con 0x55cd9d4bd800 2024-09-13T23:43:27.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd9d4ce820 con 0x55cd9d4bd800 2024-09-13T23:43:27.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd9d4ce9c0 con 0x55cd9d4bd800 2024-09-13T23:43:27.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99b90b7700 1 -- 172.21.15.167:0/2037993769 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55cd9d5b6dc0 con 0x55cd9d4bd800 2024-09-13T23:43:27.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99b90b7700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d861000 0x55cd9d7a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99b90b7700 1 -- 172.21.15.167:0/2037993769 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55cd9d523340 con 0x55cd9d4bd800 2024-09-13T23:43:27.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99bf8c4700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d861000 0x55cd9d7a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55cd9d861400 0x55cd9d7a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d861800 con 0x55cd9d861400 2024-09-13T23:43:27.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99c08c6700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55cd9d861400 0x55cd9d7a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99c08c6700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55cd9d861400 0x55cd9d7a3b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.919+0000 7f99bf8c4700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d861000 0x55cd9d7a3600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55cd9d79d920 tx=0x55cd9d5c1800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 20 realm 2024-09-13T23:43:27.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d861c00 con 0x55cd9d861400 2024-09-13T23:43:27.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:27.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d861800 con 0x55cd9d861400 2024-09-13T23:43:27.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cd9d861c00 0x55cd9d7a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8000 con 0x55cd9d861c00 2024-09-13T23:43:27.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c00c5700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cd9d861c00 0x55cd9d7a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c00c5700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cd9d861c00 0x55cd9d7a4100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:27.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861c00 2024-09-13T23:43:27.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:27.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 20 searching for the correct realm 2024-09-13T23:43:27.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55cd9d8f8800 0x55cd9d7a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.923+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8c00 con 0x55cd9d8f8800 2024-09-13T23:43:27.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.927+0000 7f99bf8c4700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55cd9d8f8800 0x55cd9d7a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.927+0000 7f99bf8c4700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55cd9d8f8800 0x55cd9d7a4680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.927+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.927+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8c00 con 0x55cd9d861c00 2024-09-13T23:43:27.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.927+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.927+0000 7f99c00c5700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55cd9d8f8000 0x55cd9d7a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.927+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8000 2024-09-13T23:43:27.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.927+0000 7f99c08c6700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55cd9d8f8000 0x55cd9d7a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.927+0000 7f99c08c6700 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55cd9d8f8000 0x55cd9d7a4c00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.927+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:27.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.927+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861400 2024-09-13T23:43:27.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861400 2024-09-13T23:43:27.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8800 2024-09-13T23:43:27.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861400 2024-09-13T23:43:27.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861c00 2024-09-13T23:43:27.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861c00 2024-09-13T23:43:27.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861400 2024-09-13T23:43:27.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.931+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8000 2024-09-13T23:43:27.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:27.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861400 2024-09-13T23:43:27.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8000 2024-09-13T23:43:27.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:27.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861c00 2024-09-13T23:43:27.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8800 2024-09-13T23:43:27.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861400 2024-09-13T23:43:27.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861c00 2024-09-13T23:43:27.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.935+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8800 2024-09-13T23:43:27.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861400 2024-09-13T23:43:27.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861400 2024-09-13T23:43:27.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861400 2024-09-13T23:43:27.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8800 2024-09-13T23:43:27.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8000 2024-09-13T23:43:27.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:27.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861400 2024-09-13T23:43:27.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.939+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8000 2024-09-13T23:43:27.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:27.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8800 2024-09-13T23:43:27.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8000 2024-09-13T23:43:27.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:27.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861c00 2024-09-13T23:43:27.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861c00 2024-09-13T23:43:27.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8800 2024-09-13T23:43:27.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.943+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861c00 2024-09-13T23:43:27.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861c00 2024-09-13T23:43:27.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c190cf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:27.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c190cf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:27.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c190cf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:27.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c190cf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:27.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9000 con 0x55cd9d861400 2024-09-13T23:43:27.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c190cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9400 con 0x55cd9d861c00 2024-09-13T23:43:27.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c190cf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:27.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.947+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9800 con 0x55cd9d861c00 2024-09-13T23:43:27.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:27.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 20 zone default found 2024-09-13T23:43:27.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 4 Realm: () 2024-09-13T23:43:27.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:27.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:27.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:27.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 20 zonegroup default 2024-09-13T23:43:27.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9c00 con 0x55cd9d8f8000 2024-09-13T23:43:27.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:27.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9000 con 0x55cd9d8f8800 2024-09-13T23:43:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9c00 con 0x55cd9d8f8800 2024-09-13T23:43:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 20 started sync module instance, tier type = 2024-09-13T23:43:27.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:27.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.951+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d861800 con 0x55cd9d861c00 2024-09-13T23:43:27.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.955+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.955+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94341598649088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8c00 con 0x55cd9d861c00 2024-09-13T23:43:27.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.955+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d861400 2024-09-13T23:43:27.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.955+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94341598649088] v33'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.955+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.955+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94341598650240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9400 con 0x55cd9d861400 2024-09-13T23:43:27.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.955+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9800 con 0x55cd9d861c00 2024-09-13T23:43:27.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.959+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94341598650240] v33'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.959+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.959+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94341598651392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9000 con 0x55cd9d861c00 2024-09-13T23:43:27.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.959+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9c00 con 0x55cd9d8f8000 2024-09-13T23:43:27.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.963+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:27.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.963+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94341598652544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d860800 con 0x55cd9d8f8000 2024-09-13T23:43:27.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.963+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d861800 con 0x55cd9d861c00 2024-09-13T23:43:27.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.963+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94341598651392] v33'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.963+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94341598652544] v33'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:27.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.963+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.963+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94341598653696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d860800 con 0x55cd9d861c00 2024-09-13T23:43:27.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.963+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8800 2024-09-13T23:43:27.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.967+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.967+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94341598653696] v33'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:27.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.967+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94341598654848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9400 con 0x55cd9d8f8800 2024-09-13T23:43:27.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.967+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9c00 con 0x55cd9d8f8800 2024-09-13T23:43:27.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.967+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94341598654848] v33'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94341598656000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8400 con 0x55cd9d8f8800 2024-09-13T23:43:27.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9400 con 0x55cd9d8f8000 2024-09-13T23:43:27.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'54 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:27.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94341598657152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9c00 con 0x55cd9d8f8000 2024-09-13T23:43:27.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 20 add_watcher() i=0 2024-09-13T23:43:27.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 20 add_watcher() i=1 2024-09-13T23:43:27.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 20 add_watcher() i=2 2024-09-13T23:43:27.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 20 add_watcher() i=3 2024-09-13T23:43:27.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 20 add_watcher() i=4 2024-09-13T23:43:27.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 20 add_watcher() i=5 2024-09-13T23:43:27.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94341598656000] v33'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:27.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 20 add_watcher() i=6 2024-09-13T23:43:27.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94341598657152] v33'55 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:27.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 20 add_watcher() i=7 2024-09-13T23:43:27.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:27.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.971+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d908000 con 0x55cd9d861400 2024-09-13T23:43:27.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.975+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.975+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94341600165888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d908400 con 0x55cd9d861400 2024-09-13T23:43:27.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.975+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94341600165888] v33'70 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.975+0000 7f99a7093700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:27.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.975+0000 7f99c190cf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:27.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.975+0000 7f99c190cf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:27.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.975+0000 7f99c190cf40 1 Processor -- start 2024-09-13T23:43:27.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.975+0000 7f99c190cf40 1 -- start start 2024-09-13T23:43:27.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.975+0000 7f99c190cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 0x55cd9d7a5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.975+0000 7f99c190cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9d54a3c0 con 0x55cd9d908c00 2024-09-13T23:43:27.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99bf8c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 0x55cd9d7a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99bf8c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 0x55cd9d7a5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51058/0 (socket says 172.21.15.167:51058) 2024-09-13T23:43:27.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2410840924 learned_addr learned my addr 172.21.15.167:0/2410840924 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2410840924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9d4ceb60 con 0x55cd9d908c00 2024-09-13T23:43:27.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99bf8c4700 1 --2- 172.21.15.167:0/2410840924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 0x55cd9d7a5180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55cd9d8a20f0 tx=0x55cd9d5c1860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a02bc290b140d3e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99a6892700 1 -- 172.21.15.167:0/2410840924 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d561760 con 0x55cd9d908c00 2024-09-13T23:43:27.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99a6892700 1 -- 172.21.15.167:0/2410840924 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd9d880900 con 0x55cd9d908c00 2024-09-13T23:43:27.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99a6892700 1 -- 172.21.15.167:0/2410840924 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d90a160 con 0x55cd9d908c00 2024-09-13T23:43:27.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99c190cf40 1 -- 172.21.15.167:0/2410840924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 msgr2=0x55cd9d7a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:27.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2410840924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 0x55cd9d7a5180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55cd9d8a20f0 tx=0x55cd9d5c1860 comp rx=0 tx=0).stop 2024-09-13T23:43:27.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99c190cf40 1 -- 172.21.15.167:0/2410840924 shutdown_connections 2024-09-13T23:43:27.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2410840924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 0x55cd9d7a5180 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:27.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99c190cf40 1 -- 172.21.15.167:0/2410840924 >> 172.21.15.167:0/2410840924 conn(0x55cd9d908800 msgr2=0x55cd9d5be000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:27.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99c190cf40 1 -- 172.21.15.167:0/2410840924 shutdown_connections 2024-09-13T23:43:27.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99c190cf40 1 -- 172.21.15.167:0/2410840924 wait complete. 2024-09-13T23:43:27.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99c190cf40 1 Processor -- start 2024-09-13T23:43:27.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99c190cf40 1 -- start start 2024-09-13T23:43:27.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99c190cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 0x55cd9d7a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99c190cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd9d54a500 con 0x55cd9d908c00 2024-09-13T23:43:27.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99bf8c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 0x55cd9d7a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99bf8c4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 0x55cd9d7a5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51070/0 (socket says 172.21.15.167:51070) 2024-09-13T23:43:27.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.979+0000 7f99bf8c4700 1 -- 172.21.15.167:0/252480356 learned_addr learned my addr 172.21.15.167:0/252480356 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:27.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99bf8c4700 1 -- 172.21.15.167:0/252480356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd9d4ced00 con 0x55cd9d908c00 2024-09-13T23:43:27.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99bf8c4700 1 --2- 172.21.15.167:0/252480356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 0x55cd9d7a5180 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55cd9d918150 tx=0x55cd9d914300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99a508f700 1 -- 172.21.15.167:0/252480356 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d90a2c0 con 0x55cd9d908c00 2024-09-13T23:43:27.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99c190cf40 1 -- 172.21.15.167:0/252480356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd9d4ce340 con 0x55cd9d908c00 2024-09-13T23:43:27.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99a508f700 1 -- 172.21.15.167:0/252480356 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd9d880600 con 0x55cd9d908c00 2024-09-13T23:43:27.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99a508f700 1 -- 172.21.15.167:0/252480356 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd9d90a420 con 0x55cd9d908c00 2024-09-13T23:43:27.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99c190cf40 1 -- 172.21.15.167:0/252480356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd9d4cfd40 con 0x55cd9d908c00 2024-09-13T23:43:27.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99a508f700 1 -- 172.21.15.167:0/252480356 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55cd9d5b7b80 con 0x55cd9d908c00 2024-09-13T23:43:27.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99a508f700 1 --2- 172.21.15.167:0/252480356 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d992000 0x55cd9d998000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:27.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99a508f700 1 -- 172.21.15.167:0/252480356 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55cd9d5236c0 con 0x55cd9d908c00 2024-09-13T23:43:27.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99c190cf40 5 note: GC not initialized 2024-09-13T23:43:27.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99c08c6700 1 --2- 172.21.15.167:0/252480356 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d992000 0x55cd9d998000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:27.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.983+0000 7f99c08c6700 1 --2- 172.21.15.167:0/252480356 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d992000 0x55cd9d998000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55cd9d8a3e00 tx=0x55cd9d81cd20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:27.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99a288a700 20 reqs_thread_entry: start 2024-09-13T23:43:27.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:27.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:27.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99a0085700 20 reqs_thread_entry: start 2024-09-13T23:43:27.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:27.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d909800 con 0x55cd9d861400 2024-09-13T23:43:27.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99a0886700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:58 2024 2024-09-13T23:43:27.997 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:27.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:27.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:27.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:27.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:27.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : miss 2024-09-13T23:43:27.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:27.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.8 8:17d68d04:users.swift::foo%3afoo1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d909c00 con 0x55cd9d861400 2024-09-13T23:43:27.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 foo:foo1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 194+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:27.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:27.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x0 2024-09-13T23:43:27.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 10 adding default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-09-13T23:43:27.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : hit (negative entry) 2024-09-13T23:43:27.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.987+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d992800 con 0x55cd9d861400 2024-09-13T23:43:28.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.991+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~513] v33'6 uv6 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:28.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.991+0000 7f99c190cf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:28.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.991+0000 7f99c190cf40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:28.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.991+0000 7f99c190cf40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:28.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.991+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94341600168192 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d992c00 con 0x55cd9d861c00 2024-09-13T23:43:28.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.991+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94341598653696 notify 141733920775 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x55cd9d4cfa00 con 0x55cd9d861c00 2024-09-13T23:43:28.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.991+0000 7f99ba8ba700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920775 cookie 94341598653696 notifier 4374 bl.length()=689 2024-09-13T23:43:28.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.991+0000 7f99ba8ba700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:28.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.991+0000 7f99ba8ba700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:28.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.991+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 notify.4 [notify cookie 94341600168192 out=8b] v0'0 uv54 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:28.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.991+0000 7f99ba8ba700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9da04000 con 0x55cd9d861c00 2024-09-13T23:43:28.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.995+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:28.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.995+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94341600168192 notify 141733920775 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55cd9d7e4000 con 0x55cd9d861c00 2024-09-13T23:43:28.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.995+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d993000 con 0x55cd9d861400 2024-09-13T23:43:28.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.995+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(67 foo:foo1 [delete,create,writefull 0~7] v33'1 uv1 ondisk = 0) v8 ==== 236+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:28.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.995+0000 7f99c190cf40 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-09-13T23:43:28.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.995+0000 7f99c190cf40 10 moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-09-13T23:43:28.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.995+0000 7f99c190cf40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-09-13T23:43:28.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.995+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94341600168192 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d993400 con 0x55cd9d861c00 2024-09-13T23:43:28.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.999+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== watch-notify(notify (1) cookie 94341598649088 notify 141733920776 ret 0) v3 ==== 208+0+0 (crc 0 0 0) 0x55cd9d7e4000 con 0x55cd9d861c00 2024-09-13T23:43:28.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.999+0000 7f99ba0b9700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920776 cookie 94341598649088 notifier 4374 bl.length()=166 2024-09-13T23:43:28.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.999+0000 7f99ba0b9700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-09-13T23:43:28.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.999+0000 7f99ba0b9700 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-09-13T23:43:28.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.999+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(68 notify.0 [notify cookie 94341600168192 out=8b] v0'0 uv54 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:28.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.999+0000 7f99ba0b9700 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f8c00 con 0x55cd9d861c00 2024-09-13T23:43:28.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.999+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== watch-notify(notify_complete (2) cookie 94341600168192 notify 141733920776 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55cd9d7e41a0 con 0x55cd9d861c00 2024-09-13T23:43:28.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.999+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:28.008 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:28.008 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:28.008 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:28.008 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:28.008 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:28.008 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [ 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "foo:foo1", 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: "permissions": "full-control" 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.009 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [ 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo:foo1", 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "ihZTiQAVEAv4Ylume3A9LC4lqJD3nM59LzvcIDYc" 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:28.010 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:28.011 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:28.012 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:28.012 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:28.012 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:28.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.999+0000 7f99a7093700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:28.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:27.999+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94341600165888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d908400 con 0x55cd9d861400 2024-09-13T23:43:28.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.003+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94341600165888] v33'71 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:28.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.003+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94341598649088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d909800 con 0x55cd9d861c00 2024-09-13T23:43:28.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.003+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94341598649088] v33'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:28.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.003+0000 7f99c190cf40 20 remove_watcher() i=0 2024-09-13T23:43:28.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.003+0000 7f99c190cf40 2 removed watcher, disabling cache 2024-09-13T23:43:28.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.003+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94341598650240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9400 con 0x55cd9d861400 2024-09-13T23:43:28.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.007+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94341598650240] v33'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d861400 2024-09-13T23:43:28.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.007+0000 7f99c190cf40 20 remove_watcher() i=1 2024-09-13T23:43:28.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.007+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94341598651392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d8f9c00 con 0x55cd9d861c00 2024-09-13T23:43:28.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.011+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94341598651392] v33'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:28.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.011+0000 7f99c190cf40 20 remove_watcher() i=2 2024-09-13T23:43:28.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.011+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94341598652544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d992800 con 0x55cd9d8f8000 2024-09-13T23:43:28.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.011+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94341598652544] v33'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:28.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.011+0000 7f99c190cf40 20 remove_watcher() i=3 2024-09-13T23:43:28.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.011+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94341598653696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d992400 con 0x55cd9d861c00 2024-09-13T23:43:28.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.015+0000 7f99c00c5700 1 -- 172.21.15.167:0/2037993769 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94341598653696] v33'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5200 con 0x55cd9d861c00 2024-09-13T23:43:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.015+0000 7f99c190cf40 20 remove_watcher() i=4 2024-09-13T23:43:28.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.015+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94341598654848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d993000 con 0x55cd9d8f8800 2024-09-13T23:43:28.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.015+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94341598654848] v33'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:28.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.015+0000 7f99c190cf40 20 remove_watcher() i=5 2024-09-13T23:43:28.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.015+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94341598656000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d993c00 con 0x55cd9d8f8800 2024-09-13T23:43:28.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.019+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94341598656000] v33'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a5440 con 0x55cd9d8f8800 2024-09-13T23:43:28.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.019+0000 7f99c190cf40 20 remove_watcher() i=6 2024-09-13T23:43:28.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.019+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94341598657152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd9d993800 con 0x55cd9d8f8000 2024-09-13T23:43:28.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c08c6700 1 -- 172.21.15.167:0/2037993769 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94341598657152] v33'56 uv54 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd9d8a4b40 con 0x55cd9d8f8000 2024-09-13T23:43:28.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 20 remove_watcher() i=7 2024-09-13T23:43:28.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cd9d861c00 msgr2=0x55cd9d7a4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cd9d861c00 0x55cd9d7a4100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55cd9d861400 msgr2=0x55cd9d7a3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55cd9d861400 0x55cd9d7a3b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55cd9d8f8000 msgr2=0x55cd9d7a4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55cd9d8f8000 0x55cd9d7a4c00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55cd9d8f8800 msgr2=0x55cd9d7a4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55cd9d8f8800 0x55cd9d7a4680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d861000 msgr2=0x55cd9d7a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d861000 0x55cd9d7a3600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x55cd9d79d920 tx=0x55cd9d5c1800 comp rx=0 tx=0).stop 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 msgr2=0x55cd9d7a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 0x55cd9d7a2b00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55cd9e32e2a0 tx=0x55cd9d81cfc0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99bf8c4700 1 -- 172.21.15.167:0/2037993769 reap_dead start 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 shutdown_connections 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/2037993769 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd800 0x55cd9d7a2b00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 >> 172.21.15.167:0/2037993769 conn(0x55cd9d4bd400 msgr2=0x55cd9d5bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 shutdown_connections 2024-09-13T23:43:28.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/2037993769 wait complete. 2024-09-13T23:43:28.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/252480356 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d992000 msgr2=0x55cd9d998000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/252480356 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d992000 0x55cd9d998000 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x55cd9d8a3e00 tx=0x55cd9d81cd20 comp rx=0 tx=0).stop 2024-09-13T23:43:28.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/252480356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 msgr2=0x55cd9d7a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/252480356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 0x55cd9d7a5180 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55cd9d918150 tx=0x55cd9d914300 comp rx=0 tx=0).stop 2024-09-13T23:43:28.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/252480356 shutdown_connections 2024-09-13T23:43:28.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/252480356 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d992000 0x55cd9d998000 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/252480356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d908c00 0x55cd9d7a5180 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/252480356 >> 172.21.15.167:0/252480356 conn(0x55cd9d908800 msgr2=0x55cd9d5be000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/252480356 shutdown_connections 2024-09-13T23:43:28.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/252480356 wait complete. 2024-09-13T23:43:28.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/3387852329 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d860000 msgr2=0x55cd9d7a3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/3387852329 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d860000 0x55cd9d7a3080 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x55cd9d619f80 tx=0x55cd9d81cde0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/3387852329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 msgr2=0x55cd9d7a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/3387852329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 0x55cd9d7a2580 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x55cd9e32e0c0 tx=0x55cd9d5c1a40 comp rx=0 tx=0).stop 2024-09-13T23:43:28.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/3387852329 shutdown_connections 2024-09-13T23:43:28.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/3387852329 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd9d860000 0x55cd9d7a3080 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 --2- 172.21.15.167:0/3387852329 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd9d4bd000 0x55cd9d7a2580 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/3387852329 >> 172.21.15.167:0/3387852329 conn(0x55cd9d4bcc00 msgr2=0x55cd9d5bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/3387852329 shutdown_connections 2024-09-13T23:43:28.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.023+0000 7f99c190cf40 1 -- 172.21.15.167:0/3387852329 wait complete. 2024-09-13T23:43:28.038 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'ihZTiQAVEAv4Ylume3A9LC4lqJD3nM59LzvcIDYc'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:28.038 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-09-13T23:43:28.038 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-09-13T23:43:28.038 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser modify --subuser foo:foo1 --secret gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL --key-type swift 2024-09-13T23:43:28.081 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:28.081 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:28.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.071+0000 7f43a2382f40 1 Processor -- start 2024-09-13T23:43:28.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.071+0000 7f43a2382f40 1 -- start start 2024-09-13T23:43:28.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.071+0000 7f43a2382f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c6778c7400 0x55c677bac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.071+0000 7f43a2382f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c67798b500 con 0x55c6778c7000 2024-09-13T23:43:28.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.071+0000 7f43a2382f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c677954a00 con 0x55c6778c7400 2024-09-13T23:43:28.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.071+0000 7f43a033a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c6778c7400 0x55c677bac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a0b3b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55c6778c7000 0x55c6779c6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45306/0 (socket says 172.21.15.167:45306) 2024-09-13T23:43:28.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a0b3b700 1 -- 172.21.15.167:0/1582633804 learned_addr learned my addr 172.21.15.167:0/1582633804 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:28.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f439fb39700 1 -- 172.21.15.167:0/1582633804 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005199695 0 0) 0x55c677a22000 con 0x55c6778c7000 2024-09-13T23:43:28.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f439fb39700 1 -- 172.21.15.167:0/1582633804 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55c67798b680 con 0x55c6778c7000 2024-09-13T23:43:28.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a033a700 1 -- 172.21.15.167:0/1582633804 >> v1:172.21.15.12:6789/0 conn(0x55c6778c7000 legacy=0x55c6779c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a033a700 1 -- 172.21.15.167:0/1582633804 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6778d9d40 con 0x55c6778c7400 2024-09-13T23:43:28.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a033a700 1 --2- 172.21.15.167:0/1582633804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7400 0x55c677bac580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55c677ba7920 tx=0x55c6779cb980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c77086963b72c51 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f439fb39700 1 -- 172.21.15.167:0/1582633804 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c67796b4a0 con 0x55c6778c7400 2024-09-13T23:43:28.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f439fb39700 1 -- 172.21.15.167:0/1582633804 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c67798b500 con 0x55c6778c7400 2024-09-13T23:43:28.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f439fb39700 1 -- 172.21.15.167:0/1582633804 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c67796b600 con 0x55c6778c7400 2024-09-13T23:43:28.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a2382f40 1 -- 172.21.15.167:0/1582633804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7400 msgr2=0x55c677bac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a2382f40 1 --2- 172.21.15.167:0/1582633804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7400 0x55c677bac580 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55c677ba7920 tx=0x55c6779cb980 comp rx=0 tx=0).stop 2024-09-13T23:43:28.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a2382f40 1 -- 172.21.15.167:0/1582633804 shutdown_connections 2024-09-13T23:43:28.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a2382f40 1 --2- 172.21.15.167:0/1582633804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7400 0x55c677bac580 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a2382f40 1 -- 172.21.15.167:0/1582633804 >> 172.21.15.167:0/1582633804 conn(0x55c6778c6c00 msgr2=0x55c6779c6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a2382f40 1 -- 172.21.15.167:0/1582633804 shutdown_connections 2024-09-13T23:43:28.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a2382f40 1 -- 172.21.15.167:0/1582633804 wait complete. 2024-09-13T23:43:28.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a2382f40 1 Processor -- start 2024-09-13T23:43:28.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a2382f40 1 -- start start 2024-09-13T23:43:28.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a2382f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 0x55c677bac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a2382f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c677954500 con 0x55c6778c7000 2024-09-13T23:43:28.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.075+0000 7f43a0b3b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 0x55c677bac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a0b3b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 0x55c677bac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51086/0 (socket says 172.21.15.167:51086) 2024-09-13T23:43:28.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a0b3b700 1 -- 172.21.15.167:0/260608097 learned_addr learned my addr 172.21.15.167:0/260608097 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a0b3b700 1 -- 172.21.15.167:0/260608097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6778d9ba0 con 0x55c6778c7000 2024-09-13T23:43:28.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a0b3b700 1 --2- 172.21.15.167:0/260608097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 0x55c677bac580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55c678691d70 tx=0x55c6779cbb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9504c63f04d5ef9c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f439f338700 1 -- 172.21.15.167:0/260608097 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c67796b4a0 con 0x55c6778c7000 2024-09-13T23:43:28.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f439f338700 1 -- 172.21.15.167:0/260608097 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c67798bb00 con 0x55c6778c7000 2024-09-13T23:43:28.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f439f338700 1 -- 172.21.15.167:0/260608097 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c67796b600 con 0x55c6778c7000 2024-09-13T23:43:28.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a2382f40 1 -- 172.21.15.167:0/260608097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 msgr2=0x55c677bac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a2382f40 1 --2- 172.21.15.167:0/260608097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 0x55c677bac580 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x55c678691d70 tx=0x55c6779cbb60 comp rx=0 tx=0).stop 2024-09-13T23:43:28.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a2382f40 1 -- 172.21.15.167:0/260608097 shutdown_connections 2024-09-13T23:43:28.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a2382f40 1 --2- 172.21.15.167:0/260608097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 0x55c677bac580 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a2382f40 1 -- 172.21.15.167:0/260608097 >> 172.21.15.167:0/260608097 conn(0x55c6778c6c00 msgr2=0x55c6779c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a2382f40 1 -- 172.21.15.167:0/260608097 shutdown_connections 2024-09-13T23:43:28.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a2382f40 1 -- 172.21.15.167:0/260608097 wait complete. 2024-09-13T23:43:28.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a2382f40 1 Processor -- start 2024-09-13T23:43:28.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a2382f40 1 -- start start 2024-09-13T23:43:28.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a2382f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 0x55c677bac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.079+0000 7f43a2382f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6779543c0 con 0x55c6778c7000 2024-09-13T23:43:28.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f43a0b3b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 0x55c677bac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f43a0b3b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 0x55c677bac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51090/0 (socket says 172.21.15.167:51090) 2024-09-13T23:43:28.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f43a0b3b700 1 -- 172.21.15.167:0/2839602308 learned_addr learned my addr 172.21.15.167:0/2839602308 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f43a0b3b700 1 -- 172.21.15.167:0/2839602308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6778d9a00 con 0x55c6778c7000 2024-09-13T23:43:28.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f43a0b3b700 1 --2- 172.21.15.167:0/2839602308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 0x55c677bac580 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55c678691e00 tx=0x55c6779cb920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f439db35700 1 -- 172.21.15.167:0/2839602308 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c67796b8c0 con 0x55c6778c7000 2024-09-13T23:43:28.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f439db35700 1 -- 172.21.15.167:0/2839602308 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c67798bc80 con 0x55c6778c7000 2024-09-13T23:43:28.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f43a2382f40 1 -- 172.21.15.167:0/2839602308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c6778d9860 con 0x55c6778c7000 2024-09-13T23:43:28.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f439db35700 1 -- 172.21.15.167:0/2839602308 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c67796b4a0 con 0x55c6778c7000 2024-09-13T23:43:28.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f43a2382f40 1 -- 172.21.15.167:0/2839602308 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6778d96c0 con 0x55c6778c7000 2024-09-13T23:43:28.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f43a2382f40 1 Processor -- start 2024-09-13T23:43:28.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f43a2382f40 1 -- start start 2024-09-13T23:43:28.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f43a2382f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 0x55c677bacb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f43a2382f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6778ec640 con 0x55c6778c7800 2024-09-13T23:43:28.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f439db35700 1 -- 172.21.15.167:0/2839602308 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c6779c0b00 con 0x55c6778c7000 2024-09-13T23:43:28.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f439db35700 1 --2- 172.21.15.167:0/2839602308 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677c68000 0x55c677bad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f439db35700 1 -- 172.21.15.167:0/2839602308 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c67792d180 con 0x55c6778c7000 2024-09-13T23:43:28.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.083+0000 7f43a133c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 0x55c677bacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a133c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 0x55c677bacb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51100/0 (socket says 172.21.15.167:51100) 2024-09-13T23:43:28.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a133c700 1 -- 172.21.15.167:0/3811431532 learned_addr learned my addr 172.21.15.167:0/3811431532 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a0b3b700 1 --2- 172.21.15.167:0/2839602308 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677c68000 0x55c677bad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a133c700 1 -- 172.21.15.167:0/3811431532 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6778d8000 con 0x55c6778c7800 2024-09-13T23:43:28.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a0b3b700 1 --2- 172.21.15.167:0/2839602308 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677c68000 0x55c677bad080 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55c677beb4d0 tx=0x55c677c22240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a133c700 1 --2- 172.21.15.167:0/3811431532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 0x55c677bacb00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55c677bea2d0 tx=0x55c677c22de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3c4b0eafd71e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f439b330700 1 -- 172.21.15.167:0/3811431532 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c67796b8c0 con 0x55c6778c7800 2024-09-13T23:43:28.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f439b330700 1 -- 172.21.15.167:0/3811431532 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c677c7c000 con 0x55c6778c7800 2024-09-13T23:43:28.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f439b330700 1 -- 172.21.15.167:0/3811431532 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c67796b4a0 con 0x55c6778c7800 2024-09-13T23:43:28.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a2382f40 1 -- 172.21.15.167:0/3811431532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 msgr2=0x55c677bacb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a2382f40 1 --2- 172.21.15.167:0/3811431532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 0x55c677bacb00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55c677bea2d0 tx=0x55c677c22de0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a2382f40 1 -- 172.21.15.167:0/3811431532 shutdown_connections 2024-09-13T23:43:28.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a2382f40 1 --2- 172.21.15.167:0/3811431532 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 0x55c677bacb00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a2382f40 1 -- 172.21.15.167:0/3811431532 >> 172.21.15.167:0/3811431532 conn(0x55c6778c7400 msgr2=0x55c6779c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a2382f40 1 -- 172.21.15.167:0/3811431532 shutdown_connections 2024-09-13T23:43:28.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a2382f40 1 -- 172.21.15.167:0/3811431532 wait complete. 2024-09-13T23:43:28.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a2382f40 1 Processor -- start 2024-09-13T23:43:28.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a2382f40 1 -- start start 2024-09-13T23:43:28.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a2382f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 0x55c677bacb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.087+0000 7f43a2382f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6778ec8c0 con 0x55c6778c7800 2024-09-13T23:43:28.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.091+0000 7f43a133c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 0x55c677bacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.091+0000 7f43a133c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 0x55c677bacb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51106/0 (socket says 172.21.15.167:51106) 2024-09-13T23:43:28.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.091+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 learned_addr learned my addr 172.21.15.167:0/488632341 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.091+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6778d8ea0 con 0x55c6778c7800 2024-09-13T23:43:28.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.091+0000 7f43a133c700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 0x55c677bacb00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55c677bea390 tx=0x55c677c22d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.091+0000 7f4399b2d700 1 -- 172.21.15.167:0/488632341 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c67796bb80 con 0x55c6778c7800 2024-09-13T23:43:28.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.091+0000 7f4399b2d700 1 -- 172.21.15.167:0/488632341 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c677c7c180 con 0x55c6778c7800 2024-09-13T23:43:28.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.091+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c6778d8820 con 0x55c6778c7800 2024-09-13T23:43:28.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.091+0000 7f4399b2d700 1 -- 172.21.15.167:0/488632341 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c67796b8c0 con 0x55c6778c7800 2024-09-13T23:43:28.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.091+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6778d89c0 con 0x55c6778c7800 2024-09-13T23:43:28.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.091+0000 7f4399b2d700 1 -- 172.21.15.167:0/488632341 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c6779c0dc0 con 0x55c6778c7800 2024-09-13T23:43:28.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.091+0000 7f43a2382f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f4399b2d700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677c68c00 0x55c677bad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f4399b2d700 1 -- 172.21.15.167:0/488632341 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c67792d340 con 0x55c6778c7800 2024-09-13T23:43:28.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a033a700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677c68c00 0x55c677bad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c677c69000 0x55c677badb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69400 con 0x55c677c69000 2024-09-13T23:43:28.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a133c700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c677c69000 0x55c677badb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a033a700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677c68c00 0x55c677bad600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55c677ba7920 tx=0x55c6779cb800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a133c700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c677c69000 0x55c677badb80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 20 realm 2024-09-13T23:43:28.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69000 2024-09-13T23:43:28.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:28.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69c00 con 0x55c677c69000 2024-09-13T23:43:28.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c677c69400 0x55c677bae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69400 2024-09-13T23:43:28.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a0b3b700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c677c69400 0x55c677bae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.095+0000 7f43a0b3b700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c677c69400 0x55c677bae100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a2382f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:28.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a2382f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69c00 con 0x55c677c69400 2024-09-13T23:43:28.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a2382f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:28.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a2382f40 20 searching for the correct realm 2024-09-13T23:43:28.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a2382f40 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c677d2e000 0x55c677bae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2e400 con 0x55c677d2e000 2024-09-13T23:43:28.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a033a700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c677d2e000 0x55c677bae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a033a700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c677d2e000 0x55c677bae680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2e400 con 0x55c677c69400 2024-09-13T23:43:28.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a0b3b700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c677d2e400 0x55c677baec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e400 2024-09-13T23:43:28.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a133c700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c677d2e400 0x55c677baec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.099+0000 7f43a133c700 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c677d2e400 0x55c677baec00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69000 2024-09-13T23:43:28.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69000 2024-09-13T23:43:28.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e000 2024-09-13T23:43:28.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69000 2024-09-13T23:43:28.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69400 2024-09-13T23:43:28.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69400 2024-09-13T23:43:28.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69000 2024-09-13T23:43:28.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.103+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e400 2024-09-13T23:43:28.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69000 2024-09-13T23:43:28.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e400 2024-09-13T23:43:28.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69400 2024-09-13T23:43:28.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e000 2024-09-13T23:43:28.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69000 2024-09-13T23:43:28.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69400 2024-09-13T23:43:28.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e000 2024-09-13T23:43:28.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.107+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69000 2024-09-13T23:43:28.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69000 2024-09-13T23:43:28.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69000 2024-09-13T23:43:28.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e000 2024-09-13T23:43:28.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e400 2024-09-13T23:43:28.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69000 2024-09-13T23:43:28.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e400 2024-09-13T23:43:28.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.111+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e000 2024-09-13T23:43:28.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.115+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.115+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e400 2024-09-13T23:43:28.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.115+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.115+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69400 2024-09-13T23:43:28.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.115+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.115+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69400 2024-09-13T23:43:28.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.115+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.115+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e000 2024-09-13T23:43:28.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.115+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.115+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69400 2024-09-13T23:43:28.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.115+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.115+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69400 2024-09-13T23:43:28.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:28.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:28.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:28.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:28.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2e800 con 0x55c677c69000 2024-09-13T23:43:28.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2ec00 con 0x55c677c69400 2024-09-13T23:43:28.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:28.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2f000 con 0x55c677c69400 2024-09-13T23:43:28.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:28.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 zone default found 2024-09-13T23:43:28.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 4 Realm: () 2024-09-13T23:43:28.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:28.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:28.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:28.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 zonegroup default 2024-09-13T23:43:28.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2f400 con 0x55c677d2e400 2024-09-13T23:43:28.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2f800 con 0x55c677d2e000 2024-09-13T23:43:28.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.119+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2fc00 con 0x55c677d2e000 2024-09-13T23:43:28.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.123+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.123+0000 7f43a2382f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.123+0000 7f43a2382f40 20 started sync module instance, tier type = 2024-09-13T23:43:28.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.123+0000 7f43a2382f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:28.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.123+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2f800 con 0x55c677c69400 2024-09-13T23:43:28.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.123+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.123+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94310900579072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2fc00 con 0x55c677c69400 2024-09-13T23:43:28.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.123+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2ec00 con 0x55c677c69000 2024-09-13T23:43:28.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.127+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.127+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94310900580224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2f000 con 0x55c677c69000 2024-09-13T23:43:28.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.127+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2f800 con 0x55c677c69400 2024-09-13T23:43:28.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.127+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94310900579072] v33'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.131+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94310900580224] v33'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.131+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.131+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94310900581376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69c00 con 0x55c677c69400 2024-09-13T23:43:28.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.131+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677d2e400 2024-09-13T23:43:28.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.131+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.131+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94310900582528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2e800 con 0x55c677d2e400 2024-09-13T23:43:28.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.131+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2f400 con 0x55c677c69400 2024-09-13T23:43:28.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.131+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94310900581376] v33'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.135+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94310900582528] v33'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.135+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.135+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94310900583680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2ec00 con 0x55c677c69400 2024-09-13T23:43:28.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.135+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2f000 con 0x55c677d2e000 2024-09-13T23:43:28.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.135+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.135+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94310900584832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d38000 con 0x55c677d2e000 2024-09-13T23:43:28.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.135+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d38400 con 0x55c677d2e000 2024-09-13T23:43:28.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.139+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94310900583680] v33'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.139+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94310900584832] v33'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.139+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.139+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94310900585984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d38800 con 0x55c677d2e000 2024-09-13T23:43:28.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.139+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d38c00 con 0x55c677d2e400 2024-09-13T23:43:28.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.139+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94310900585984] v33'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'57 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94310900587136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d39000 con 0x55c677d2e400 2024-09-13T23:43:28.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a2382f40 20 add_watcher() i=0 2024-09-13T23:43:28.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a2382f40 20 add_watcher() i=1 2024-09-13T23:43:28.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a2382f40 20 add_watcher() i=2 2024-09-13T23:43:28.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a2382f40 20 add_watcher() i=3 2024-09-13T23:43:28.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a2382f40 20 add_watcher() i=4 2024-09-13T23:43:28.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a2382f40 20 add_watcher() i=5 2024-09-13T23:43:28.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a2382f40 20 add_watcher() i=6 2024-09-13T23:43:28.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94310900587136] v33'58 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a2382f40 20 add_watcher() i=7 2024-09-13T23:43:28.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a2382f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:28.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d39400 con 0x55c677c69000 2024-09-13T23:43:28.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.143+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94310902259712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d39800 con 0x55c677c69000 2024-09-13T23:43:28.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94310902259712] v33'72 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f4387b09700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:28.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f43a2382f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:28.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f43a2382f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:28.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f43a2382f40 1 Processor -- start 2024-09-13T23:43:28.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f43a2382f40 1 -- start start 2024-09-13T23:43:28.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f43a2382f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 0x55c677baf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f43a2382f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6779543c0 con 0x55c677d38c00 2024-09-13T23:43:28.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f43a033a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 0x55c677baf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f43a033a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 0x55c677baf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51118/0 (socket says 172.21.15.167:51118) 2024-09-13T23:43:28.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f43a033a700 1 -- 172.21.15.167:0/3603889311 learned_addr learned my addr 172.21.15.167:0/3603889311 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f43a033a700 1 -- 172.21.15.167:0/3603889311 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6778d8b60 con 0x55c677d38c00 2024-09-13T23:43:28.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f43a033a700 1 --2- 172.21.15.167:0/3603889311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 0x55c677baf180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55c677cc37a0 tx=0x55c6779cb860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe58af56e7796e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f4387308700 1 -- 172.21.15.167:0/3603889311 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c67796b760 con 0x55c677d38c00 2024-09-13T23:43:28.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f4387308700 1 -- 172.21.15.167:0/3603889311 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c677c7c600 con 0x55c677d38c00 2024-09-13T23:43:28.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.147+0000 7f4387308700 1 -- 172.21.15.167:0/3603889311 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c677d52000 con 0x55c677d38c00 2024-09-13T23:43:28.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 -- 172.21.15.167:0/3603889311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 msgr2=0x55c677baf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 --2- 172.21.15.167:0/3603889311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 0x55c677baf180 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55c677cc37a0 tx=0x55c6779cb860 comp rx=0 tx=0).stop 2024-09-13T23:43:28.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 -- 172.21.15.167:0/3603889311 shutdown_connections 2024-09-13T23:43:28.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 --2- 172.21.15.167:0/3603889311 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 0x55c677baf180 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 -- 172.21.15.167:0/3603889311 >> 172.21.15.167:0/3603889311 conn(0x55c677d39c00 msgr2=0x55c6779c8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 -- 172.21.15.167:0/3603889311 shutdown_connections 2024-09-13T23:43:28.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 -- 172.21.15.167:0/3603889311 wait complete. 2024-09-13T23:43:28.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 Processor -- start 2024-09-13T23:43:28.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 -- start start 2024-09-13T23:43:28.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 0x55c677baf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c677954500 con 0x55c677d38c00 2024-09-13T23:43:28.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a033a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 0x55c677baf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a033a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 0x55c677baf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51120/0 (socket says 172.21.15.167:51120) 2024-09-13T23:43:28.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a033a700 1 -- 172.21.15.167:0/1512159319 learned_addr learned my addr 172.21.15.167:0/1512159319 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a033a700 1 -- 172.21.15.167:0/1512159319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6778d8d00 con 0x55c677d38c00 2024-09-13T23:43:28.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a033a700 1 --2- 172.21.15.167:0/1512159319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 0x55c677baf180 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55c677d42600 tx=0x55c677d4a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f4385b05700 1 -- 172.21.15.167:0/1512159319 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c677d52160 con 0x55c677d38c00 2024-09-13T23:43:28.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f4385b05700 1 -- 172.21.15.167:0/1512159319 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c677c7c300 con 0x55c677d38c00 2024-09-13T23:43:28.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 -- 172.21.15.167:0/1512159319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c6778d8340 con 0x55c677d38c00 2024-09-13T23:43:28.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f4385b05700 1 -- 172.21.15.167:0/1512159319 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c677d522c0 con 0x55c677d38c00 2024-09-13T23:43:28.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.151+0000 7f43a2382f40 1 -- 172.21.15.167:0/1512159319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6778d9d40 con 0x55c677d38c00 2024-09-13T23:43:28.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f4385b05700 1 -- 172.21.15.167:0/1512159319 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c6779c1b80 con 0x55c677d38c00 2024-09-13T23:43:28.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f4385b05700 1 --2- 172.21.15.167:0/1512159319 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677dc6000 0x55c677dcc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f4385b05700 1 -- 172.21.15.167:0/1512159319 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c67792d500 con 0x55c677d38c00 2024-09-13T23:43:28.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f43a2382f40 5 note: GC not initialized 2024-09-13T23:43:28.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f43a133c700 1 --2- 172.21.15.167:0/1512159319 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677dc6000 0x55c677dcc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f43a133c700 1 --2- 172.21.15.167:0/1512159319 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677dc6000 0x55c677dcc000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55c677cc3dd0 tx=0x55c677c237a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f4383300700 20 reqs_thread_entry: start 2024-09-13T23:43:28.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f43a2382f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:28.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f43a2382f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:28.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f43812fc700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:58 2024 2024-09-13T23:43:28.166 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:28.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f43a2382f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:28.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f43a2382f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f4380ada700 20 reqs_thread_entry: start 2024-09-13T23:43:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.155+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d38000 con 0x55c677c69000 2024-09-13T23:43:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.159+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv6 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.159+0000 7f43a2382f40 20 rados_obj.operate() r=0 bl.length=513 2024-09-13T23:43:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.159+0000 7f43a2382f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.159+0000 7f43a2382f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:28.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.159+0000 7f43a2382f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:28.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.159+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2e800 con 0x55c677c69000 2024-09-13T23:43:28.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.159+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~513] v33'7 uv7 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.159+0000 7f43a2382f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:28.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.159+0000 7f43a2382f40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:28.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.159+0000 7f43a2382f40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:28.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.159+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94310902262016 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2f000 con 0x55c677c69400 2024-09-13T23:43:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.163+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94310900583680 notify 141733920777 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x55c677c20820 con 0x55c677c69400 2024-09-13T23:43:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.163+0000 7f439b330700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920777 cookie 94310900583680 notifier 4382 bl.length()=689 2024-09-13T23:43:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.163+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(64 notify.4 [notify cookie 94310902262016 out=8b] v0'0 uv57 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.163+0000 7f439b330700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.163+0000 7f439b330700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.163+0000 7f439b330700 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2fc00 con 0x55c677c69400 2024-09-13T23:43:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.163+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== watch-notify(notify_complete (2) cookie 94310902262016 notify 141733920777 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55c677c209c0 con 0x55c677c69400 2024-09-13T23:43:28.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.163+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.172 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [ 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "foo:foo1", 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: "permissions": "full-control" 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:28.173 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [ 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo:foo1", 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:28.174 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:28.175 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:28.176 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:28.176 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.176 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:28.176 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:28.176 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:28.176 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:28.176 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:28.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.163+0000 7f4387b09700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:28.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.163+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94310902259712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677c69800 con 0x55c677c69000 2024-09-13T23:43:28.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.167+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94310902259712] v33'73 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.167+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94310900579072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677dc6400 con 0x55c677c69400 2024-09-13T23:43:28.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.171+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94310900579072] v33'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.171+0000 7f43a2382f40 20 remove_watcher() i=0 2024-09-13T23:43:28.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.171+0000 7f43a2382f40 2 removed watcher, disabling cache 2024-09-13T23:43:28.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.171+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94310900580224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677dc6800 con 0x55c677c69000 2024-09-13T23:43:28.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.171+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94310900580224] v33'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677c69000 2024-09-13T23:43:28.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.171+0000 7f43a2382f40 20 remove_watcher() i=1 2024-09-13T23:43:28.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.171+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94310900581376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677dc6c00 con 0x55c677c69400 2024-09-13T23:43:28.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.175+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94310900581376] v33'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.175+0000 7f43a2382f40 20 remove_watcher() i=2 2024-09-13T23:43:28.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.175+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94310900582528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677dc7000 con 0x55c677d2e400 2024-09-13T23:43:28.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.179+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94310900582528] v33'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.179+0000 7f43a2382f40 20 remove_watcher() i=3 2024-09-13T23:43:28.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.179+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94310900583680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677dc7400 con 0x55c677c69400 2024-09-13T23:43:28.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.179+0000 7f43a0b3b700 1 -- 172.21.15.167:0/488632341 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94310900583680] v33'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17200 con 0x55c677c69400 2024-09-13T23:43:28.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.179+0000 7f43a2382f40 20 remove_watcher() i=4 2024-09-13T23:43:28.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.179+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94310900584832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677dc7800 con 0x55c677d2e000 2024-09-13T23:43:28.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.183+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94310900584832] v33'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.183+0000 7f43a2382f40 20 remove_watcher() i=5 2024-09-13T23:43:28.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.183+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94310900585984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677dc7c00 con 0x55c677d2e000 2024-09-13T23:43:28.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.183+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94310900585984] v33'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d17440 con 0x55c677d2e000 2024-09-13T23:43:28.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.183+0000 7f43a2382f40 20 remove_watcher() i=6 2024-09-13T23:43:28.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.183+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94310900587136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c677d2f800 con 0x55c677d2e400 2024-09-13T23:43:28.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a133c700 1 -- 172.21.15.167:0/488632341 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94310900587136] v33'59 uv57 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c677d16b40 con 0x55c677d2e400 2024-09-13T23:43:28.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 20 remove_watcher() i=7 2024-09-13T23:43:28.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c677c69400 msgr2=0x55c677bae100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c677c69400 0x55c677bae100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c677c69000 msgr2=0x55c677badb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c677c69000 0x55c677badb80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c677d2e400 msgr2=0x55c677baec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c677d2e400 0x55c677baec00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c677d2e000 msgr2=0x55c677bae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c677d2e000 0x55c677bae680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677c68c00 msgr2=0x55c677bad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677c68c00 0x55c677bad600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55c677ba7920 tx=0x55c6779cb800 comp rx=0 tx=0).stop 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 msgr2=0x55c677bacb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 0x55c677bacb00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55c677bea390 tx=0x55c677c22d20 comp rx=0 tx=0).stop 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a033a700 1 -- 172.21.15.167:0/488632341 reap_dead start 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 shutdown_connections 2024-09-13T23:43:28.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 --2- 172.21.15.167:0/488632341 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7800 0x55c677bacb00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 >> 172.21.15.167:0/488632341 conn(0x55c6778c7400 msgr2=0x55c6779c7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 shutdown_connections 2024-09-13T23:43:28.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 -- 172.21.15.167:0/488632341 wait complete. 2024-09-13T23:43:28.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 -- 172.21.15.167:0/1512159319 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677dc6000 msgr2=0x55c677dcc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 --2- 172.21.15.167:0/1512159319 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677dc6000 0x55c677dcc000 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55c677cc3dd0 tx=0x55c677c237a0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 -- 172.21.15.167:0/1512159319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 msgr2=0x55c677baf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.187+0000 7f43a2382f40 1 --2- 172.21.15.167:0/1512159319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 0x55c677baf180 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55c677d42600 tx=0x55c677d4a300 comp rx=0 tx=0).stop 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 -- 172.21.15.167:0/1512159319 shutdown_connections 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 --2- 172.21.15.167:0/1512159319 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677dc6000 0x55c677dcc000 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 --2- 172.21.15.167:0/1512159319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c677d38c00 0x55c677baf180 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 -- 172.21.15.167:0/1512159319 >> 172.21.15.167:0/1512159319 conn(0x55c677d39c00 msgr2=0x55c6779c8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 -- 172.21.15.167:0/1512159319 shutdown_connections 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 -- 172.21.15.167:0/1512159319 wait complete. 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 -- 172.21.15.167:0/2839602308 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677c68000 msgr2=0x55c677bad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 --2- 172.21.15.167:0/2839602308 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677c68000 0x55c677bad080 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55c677beb4d0 tx=0x55c677c22240 comp rx=0 tx=0).stop 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 -- 172.21.15.167:0/2839602308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 msgr2=0x55c677bac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 --2- 172.21.15.167:0/2839602308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 0x55c677bac580 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x55c678691e00 tx=0x55c6779cb920 comp rx=0 tx=0).stop 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 -- 172.21.15.167:0/2839602308 shutdown_connections 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 --2- 172.21.15.167:0/2839602308 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c677c68000 0x55c677bad080 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 --2- 172.21.15.167:0/2839602308 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c6778c7000 0x55c677bac580 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 -- 172.21.15.167:0/2839602308 >> 172.21.15.167:0/2839602308 conn(0x55c6778c6c00 msgr2=0x55c6779c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 -- 172.21.15.167:0/2839602308 shutdown_connections 2024-09-13T23:43:28.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.191+0000 7f43a2382f40 1 -- 172.21.15.167:0/2839602308 wait complete. 2024-09-13T23:43:28.205 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:28.206 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-13T23:43:28.206 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-13T23:43:28.206 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-13T23:43:28.246 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:28.247 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:28.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee47dcaf40 1 Processor -- start 2024-09-13T23:43:28.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee47dcaf40 1 -- start start 2024-09-13T23:43:28.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee47dcaf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55cd16a03400 0x55cd16ce8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee47dcaf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55cd16ac7500 con 0x55cd16a03000 2024-09-13T23:43:28.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee47dcaf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55cd16a90a00 con 0x55cd16a03400 2024-09-13T23:43:28.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee45d82700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55cd16a03400 0x55cd16ce8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee46583700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55cd16a03000 0x55cd16b02800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45308/0 (socket says 172.21.15.167:45308) 2024-09-13T23:43:28.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee45d82700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55cd16a03400 0x55cd16ce8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51132/0 (socket says 172.21.15.167:51132) 2024-09-13T23:43:28.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee46583700 1 -- 172.21.15.167:0/1961141215 learned_addr learned my addr 172.21.15.167:0/1961141215 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:28.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee45d82700 1 -- 172.21.15.167:0/1961141215 >> v1:172.21.15.12:6789/0 conn(0x55cd16a03000 legacy=0x55cd16b02800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee45d82700 1 -- 172.21.15.167:0/1961141215 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd16a15d40 con 0x55cd16a03400 2024-09-13T23:43:28.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee45d82700 1 --2- 172.21.15.167:0/1961141215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03400 0x55cd16ce8580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55cd16ce3920 tx=0x55cd16b07920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e07b9a3423bcb39 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee45581700 1 -- 172.21.15.167:0/1961141215 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16aa74a0 con 0x55cd16a03400 2024-09-13T23:43:28.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee45581700 1 -- 172.21.15.167:0/1961141215 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd16ac7800 con 0x55cd16a03400 2024-09-13T23:43:28.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee45581700 1 -- 172.21.15.167:0/1961141215 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16aa7600 con 0x55cd16a03400 2024-09-13T23:43:28.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1961141215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03400 msgr2=0x55cd16ce8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/1961141215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03400 0x55cd16ce8580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55cd16ce3920 tx=0x55cd16b07920 comp rx=0 tx=0).stop 2024-09-13T23:43:28.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1961141215 shutdown_connections 2024-09-13T23:43:28.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/1961141215 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03400 0x55cd16ce8580 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1961141215 >> 172.21.15.167:0/1961141215 conn(0x55cd16a02c00 msgr2=0x55cd16b02000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1961141215 shutdown_connections 2024-09-13T23:43:28.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.239+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1961141215 wait complete. 2024-09-13T23:43:28.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 Processor -- start 2024-09-13T23:43:28.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 -- start start 2024-09-13T23:43:28.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 0x55cd16ce8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd16a90500 con 0x55cd16a03000 2024-09-13T23:43:28.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee46583700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 0x55cd16ce8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee46583700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 0x55cd16ce8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51138/0 (socket says 172.21.15.167:51138) 2024-09-13T23:43:28.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee46583700 1 -- 172.21.15.167:0/635556594 learned_addr learned my addr 172.21.15.167:0/635556594 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee46583700 1 -- 172.21.15.167:0/635556594 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd16a15ba0 con 0x55cd16a03000 2024-09-13T23:43:28.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee46583700 1 --2- 172.21.15.167:0/635556594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 0x55cd16ce8580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55cd16b5fe60 tx=0x55cd16b07b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf83ff6fe7f36ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee44d80700 1 -- 172.21.15.167:0/635556594 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16aa74a0 con 0x55cd16a03000 2024-09-13T23:43:28.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee44d80700 1 -- 172.21.15.167:0/635556594 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd16ac7b00 con 0x55cd16a03000 2024-09-13T23:43:28.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee44d80700 1 -- 172.21.15.167:0/635556594 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16aa7600 con 0x55cd16a03000 2024-09-13T23:43:28.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 -- 172.21.15.167:0/635556594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 msgr2=0x55cd16ce8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/635556594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 0x55cd16ce8580 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55cd16b5fe60 tx=0x55cd16b07b60 comp rx=0 tx=0).stop 2024-09-13T23:43:28.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 -- 172.21.15.167:0/635556594 shutdown_connections 2024-09-13T23:43:28.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/635556594 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 0x55cd16ce8580 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 -- 172.21.15.167:0/635556594 >> 172.21.15.167:0/635556594 conn(0x55cd16a02c00 msgr2=0x55cd16b03000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 -- 172.21.15.167:0/635556594 shutdown_connections 2024-09-13T23:43:28.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 -- 172.21.15.167:0/635556594 wait complete. 2024-09-13T23:43:28.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 Processor -- start 2024-09-13T23:43:28.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 -- start start 2024-09-13T23:43:28.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 0x55cd16ce8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.243+0000 7fee47dcaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd16a903c0 con 0x55cd16a03000 2024-09-13T23:43:28.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee46583700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 0x55cd16ce8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee46583700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 0x55cd16ce8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51144/0 (socket says 172.21.15.167:51144) 2024-09-13T23:43:28.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee46583700 1 -- 172.21.15.167:0/3063374043 learned_addr learned my addr 172.21.15.167:0/3063374043 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee46583700 1 -- 172.21.15.167:0/3063374043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd16a15a00 con 0x55cd16a03000 2024-09-13T23:43:28.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee46583700 1 --2- 172.21.15.167:0/3063374043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 0x55cd16ce8580 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55cd16b5fec0 tx=0x55cd16b07a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee4357d700 1 -- 172.21.15.167:0/3063374043 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16aa78c0 con 0x55cd16a03000 2024-09-13T23:43:28.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee4357d700 1 -- 172.21.15.167:0/3063374043 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd16ac7c80 con 0x55cd16a03000 2024-09-13T23:43:28.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3063374043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd16a15860 con 0x55cd16a03000 2024-09-13T23:43:28.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee4357d700 1 -- 172.21.15.167:0/3063374043 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16aa74a0 con 0x55cd16a03000 2024-09-13T23:43:28.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3063374043 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd16a156c0 con 0x55cd16a03000 2024-09-13T23:43:28.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee47dcaf40 1 Processor -- start 2024-09-13T23:43:28.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee47dcaf40 1 -- start start 2024-09-13T23:43:28.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee47dcaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 0x55cd16ce8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee47dcaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd16a90000 con 0x55cd16a03800 2024-09-13T23:43:28.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee4357d700 1 -- 172.21.15.167:0/3063374043 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55cd16afcb00 con 0x55cd16a03000 2024-09-13T23:43:28.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee4357d700 1 --2- 172.21.15.167:0/3063374043 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16db0000 0x55cd16ce9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee4357d700 1 -- 172.21.15.167:0/3063374043 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55cd16a69180 con 0x55cd16a03000 2024-09-13T23:43:28.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.247+0000 7fee46d84700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 0x55cd16ce8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee46d84700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 0x55cd16ce8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51156/0 (socket says 172.21.15.167:51156) 2024-09-13T23:43:28.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee46d84700 1 -- 172.21.15.167:0/3217146557 learned_addr learned my addr 172.21.15.167:0/3217146557 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee46583700 1 --2- 172.21.15.167:0/3063374043 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16db0000 0x55cd16ce9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee46d84700 1 -- 172.21.15.167:0/3217146557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd16a14000 con 0x55cd16a03800 2024-09-13T23:43:28.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee46583700 1 --2- 172.21.15.167:0/3063374043 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16db0000 0x55cd16ce9080 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55cd1787b5c0 tx=0x55cd16d66240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee46d84700 1 --2- 172.21.15.167:0/3217146557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 0x55cd16ce8b00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55cd1787a300 tx=0x55cd16d670e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d18edd76aeac8c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee40d78700 1 -- 172.21.15.167:0/3217146557 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16aa78c0 con 0x55cd16a03800 2024-09-13T23:43:28.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee40d78700 1 -- 172.21.15.167:0/3217146557 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd16dc2000 con 0x55cd16a03800 2024-09-13T23:43:28.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee40d78700 1 -- 172.21.15.167:0/3217146557 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16aa74a0 con 0x55cd16a03800 2024-09-13T23:43:28.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3217146557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 msgr2=0x55cd16ce8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/3217146557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 0x55cd16ce8b00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55cd1787a300 tx=0x55cd16d670e0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3217146557 shutdown_connections 2024-09-13T23:43:28.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/3217146557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 0x55cd16ce8b00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3217146557 >> 172.21.15.167:0/3217146557 conn(0x55cd16a03400 msgr2=0x55cd16b03800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3217146557 shutdown_connections 2024-09-13T23:43:28.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3217146557 wait complete. 2024-09-13T23:43:28.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee47dcaf40 1 Processor -- start 2024-09-13T23:43:28.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee47dcaf40 1 -- start start 2024-09-13T23:43:28.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee47dcaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 0x55cd16ce8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee47dcaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd16a28640 con 0x55cd16a03800 2024-09-13T23:43:28.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee46d84700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 0x55cd16ce8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee46d84700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 0x55cd16ce8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51172/0 (socket says 172.21.15.167:51172) 2024-09-13T23:43:28.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 learned_addr learned my addr 172.21.15.167:0/2613476649 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.251+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd16a14ea0 con 0x55cd16a03800 2024-09-13T23:43:28.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee46d84700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 0x55cd16ce8b00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55cd1787a3f0 tx=0x55cd16d66cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee3f575700 1 -- 172.21.15.167:0/2613476649 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16aa7b80 con 0x55cd16a03800 2024-09-13T23:43:28.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee3f575700 1 -- 172.21.15.167:0/2613476649 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd16dc2180 con 0x55cd16a03800 2024-09-13T23:43:28.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd16a14820 con 0x55cd16a03800 2024-09-13T23:43:28.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee3f575700 1 -- 172.21.15.167:0/2613476649 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16aa78c0 con 0x55cd16a03800 2024-09-13T23:43:28.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd16a149c0 con 0x55cd16a03800 2024-09-13T23:43:28.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee47dcaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee3f575700 1 -- 172.21.15.167:0/2613476649 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55cd16afcdc0 con 0x55cd16a03800 2024-09-13T23:43:28.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee3f575700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16db0c00 0x55cd16ce9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee3f575700 1 -- 172.21.15.167:0/2613476649 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55cd16a69340 con 0x55cd16a03800 2024-09-13T23:43:28.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee45d82700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16db0c00 0x55cd16ce9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55cd16db1000 0x55cd16ce9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1400 con 0x55cd16db1000 2024-09-13T23:43:28.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee46d84700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55cd16db1000 0x55cd16ce9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee45d82700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16db0c00 0x55cd16ce9600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55cd16ce3920 tx=0x55cd16b07800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.255+0000 7fee46d84700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55cd16db1000 0x55cd16ce9b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 20 realm 2024-09-13T23:43:28.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1000 2024-09-13T23:43:28.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:28.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1c00 con 0x55cd16db1000 2024-09-13T23:43:28.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cd16db1400 0x55cd16cea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1400 2024-09-13T23:43:28.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee46583700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cd16db1400 0x55cd16cea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee46583700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cd16db1400 0x55cd16cea100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:28.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1c00 con 0x55cd16db1400 2024-09-13T23:43:28.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:28.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 20 searching for the correct realm 2024-09-13T23:43:28.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55cd16d4c000 0x55cd16cea680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4c400 con 0x55cd16d4c000 2024-09-13T23:43:28.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.259+0000 7fee45d82700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55cd16d4c000 0x55cd16cea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.263+0000 7fee45d82700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55cd16d4c000 0x55cd16cea680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.263+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.263+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4c400 con 0x55cd16db1400 2024-09-13T23:43:28.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.263+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.263+0000 7fee46583700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55cd16d4c400 0x55cd16ceac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.263+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c400 2024-09-13T23:43:28.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.263+0000 7fee46d84700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55cd16d4c400 0x55cd16ceac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.263+0000 7fee46d84700 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55cd16d4c400 0x55cd16ceac00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.263+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.263+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1000 2024-09-13T23:43:28.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.263+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.263+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1000 2024-09-13T23:43:28.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c000 2024-09-13T23:43:28.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1000 2024-09-13T23:43:28.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1400 2024-09-13T23:43:28.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1400 2024-09-13T23:43:28.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1000 2024-09-13T23:43:28.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c400 2024-09-13T23:43:28.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.267+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1000 2024-09-13T23:43:28.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c400 2024-09-13T23:43:28.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1400 2024-09-13T23:43:28.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c000 2024-09-13T23:43:28.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1000 2024-09-13T23:43:28.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1400 2024-09-13T23:43:28.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c000 2024-09-13T23:43:28.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.271+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1000 2024-09-13T23:43:28.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1000 2024-09-13T23:43:28.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1000 2024-09-13T23:43:28.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c000 2024-09-13T23:43:28.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c400 2024-09-13T23:43:28.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1000 2024-09-13T23:43:28.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c400 2024-09-13T23:43:28.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.275+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c000 2024-09-13T23:43:28.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.279+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.279+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c400 2024-09-13T23:43:28.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.279+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.279+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1400 2024-09-13T23:43:28.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.279+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.279+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1400 2024-09-13T23:43:28.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.279+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.279+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c000 2024-09-13T23:43:28.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.279+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.279+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1400 2024-09-13T23:43:28.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.279+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.279+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1400 2024-09-13T23:43:28.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:28.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:28.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:28.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:28.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4c800 con 0x55cd16db1000 2024-09-13T23:43:28.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4cc00 con 0x55cd16db1400 2024-09-13T23:43:28.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:28.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4d000 con 0x55cd16db1400 2024-09-13T23:43:28.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:28.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 zone default found 2024-09-13T23:43:28.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 4 Realm: () 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 zonegroup default 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4d400 con 0x55cd16d4c400 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.283+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4d800 con 0x55cd16d4c000 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.287+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.287+0000 7fee47dcaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.287+0000 7fee47dcaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.287+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4dc00 con 0x55cd16d4c000 2024-09-13T23:43:28.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.287+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.287+0000 7fee47dcaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.287+0000 7fee47dcaf40 20 started sync module instance, tier type = 2024-09-13T23:43:28.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.287+0000 7fee47dcaf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:28.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.287+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4d800 con 0x55cd16db1400 2024-09-13T23:43:28.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.287+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.287+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94339339254528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4dc00 con 0x55cd16db1400 2024-09-13T23:43:28.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.291+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4cc00 con 0x55cd16db1000 2024-09-13T23:43:28.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.291+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94339339254528] v33'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.291+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.291+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94339339255680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4d000 con 0x55cd16db1000 2024-09-13T23:43:28.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.291+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4d800 con 0x55cd16db1400 2024-09-13T23:43:28.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.295+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94339339255680] v33'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.295+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.295+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94339339256832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1c00 con 0x55cd16db1400 2024-09-13T23:43:28.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.295+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16d4c400 2024-09-13T23:43:28.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.299+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94339339256832] v33'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.299+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.299+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94339339257984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4c800 con 0x55cd16d4c400 2024-09-13T23:43:28.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.299+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4d400 con 0x55cd16db1400 2024-09-13T23:43:28.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.299+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94339339257984] v33'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.299+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.299+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94339339259136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4cc00 con 0x55cd16db1400 2024-09-13T23:43:28.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.299+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4d000 con 0x55cd16d4c000 2024-09-13T23:43:28.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.303+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.303+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94339339260288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16e7a000 con 0x55cd16d4c000 2024-09-13T23:43:28.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.303+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16e7a400 con 0x55cd16d4c000 2024-09-13T23:43:28.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.303+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94339339259136] v33'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.303+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94339339260288] v33'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94339339261440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16e7a800 con 0x55cd16d4c000 2024-09-13T23:43:28.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16e7ac00 con 0x55cd16d4c400 2024-09-13T23:43:28.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'60 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94339339262592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16e7b000 con 0x55cd16d4c400 2024-09-13T23:43:28.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee47dcaf40 20 add_watcher() i=0 2024-09-13T23:43:28.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee47dcaf40 20 add_watcher() i=1 2024-09-13T23:43:28.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee47dcaf40 20 add_watcher() i=2 2024-09-13T23:43:28.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee47dcaf40 20 add_watcher() i=3 2024-09-13T23:43:28.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee47dcaf40 20 add_watcher() i=4 2024-09-13T23:43:28.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee47dcaf40 20 add_watcher() i=5 2024-09-13T23:43:28.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94339339261440] v33'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.307+0000 7fee47dcaf40 20 add_watcher() i=6 2024-09-13T23:43:28.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94339339262592] v33'61 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee47dcaf40 20 add_watcher() i=7 2024-09-13T23:43:28.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee47dcaf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:28.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16e7b400 con 0x55cd16db1000 2024-09-13T23:43:28.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94339340959744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16e7b800 con 0x55cd16db1000 2024-09-13T23:43:28.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94339340959744] v33'74 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee2d551700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:28.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee47dcaf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:28.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee47dcaf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:28.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee47dcaf40 1 Processor -- start 2024-09-13T23:43:28.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee47dcaf40 1 -- start start 2024-09-13T23:43:28.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee47dcaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 0x55cd16ceb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.311+0000 7fee47dcaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd16a28780 con 0x55cd16e7ac00 2024-09-13T23:43:28.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee45d82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 0x55cd16ceb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee45d82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 0x55cd16ceb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51188/0 (socket says 172.21.15.167:51188) 2024-09-13T23:43:28.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee45d82700 1 -- 172.21.15.167:0/225877097 learned_addr learned my addr 172.21.15.167:0/225877097 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee45d82700 1 -- 172.21.15.167:0/225877097 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd16a14b60 con 0x55cd16e7ac00 2024-09-13T23:43:28.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee45d82700 1 --2- 172.21.15.167:0/225877097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 0x55cd16ceb180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55cd16b5e0c0 tx=0x55cd16b07860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96188d51736bf390 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee2cd50700 1 -- 172.21.15.167:0/225877097 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16aa7760 con 0x55cd16e7ac00 2024-09-13T23:43:28.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee2cd50700 1 -- 172.21.15.167:0/225877097 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd16dc2600 con 0x55cd16e7ac00 2024-09-13T23:43:28.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee2cd50700 1 -- 172.21.15.167:0/225877097 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16e94000 con 0x55cd16e7ac00 2024-09-13T23:43:28.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee47dcaf40 1 -- 172.21.15.167:0/225877097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 msgr2=0x55cd16ceb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/225877097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 0x55cd16ceb180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x55cd16b5e0c0 tx=0x55cd16b07860 comp rx=0 tx=0).stop 2024-09-13T23:43:28.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee47dcaf40 1 -- 172.21.15.167:0/225877097 shutdown_connections 2024-09-13T23:43:28.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/225877097 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 0x55cd16ceb180 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee47dcaf40 1 -- 172.21.15.167:0/225877097 >> 172.21.15.167:0/225877097 conn(0x55cd16e7bc00 msgr2=0x55cd16b04000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee47dcaf40 1 -- 172.21.15.167:0/225877097 shutdown_connections 2024-09-13T23:43:28.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee47dcaf40 1 -- 172.21.15.167:0/225877097 wait complete. 2024-09-13T23:43:28.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee47dcaf40 1 Processor -- start 2024-09-13T23:43:28.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee47dcaf40 1 -- start start 2024-09-13T23:43:28.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee47dcaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 0x55cd16ceb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.315+0000 7fee47dcaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd16a90000 con 0x55cd16e7ac00 2024-09-13T23:43:28.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee45d82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 0x55cd16ceb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee45d82700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 0x55cd16ceb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51202/0 (socket says 172.21.15.167:51202) 2024-09-13T23:43:28.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee45d82700 1 -- 172.21.15.167:0/1928088351 learned_addr learned my addr 172.21.15.167:0/1928088351 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee45d82700 1 -- 172.21.15.167:0/1928088351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd16a14d00 con 0x55cd16e7ac00 2024-09-13T23:43:28.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee45d82700 1 --2- 172.21.15.167:0/1928088351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 0x55cd16ceb180 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55cd16e92150 tx=0x55cd16e8a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee2b54d700 1 -- 172.21.15.167:0/1928088351 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16e94160 con 0x55cd16e7ac00 2024-09-13T23:43:28.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee2b54d700 1 -- 172.21.15.167:0/1928088351 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cd16dc2300 con 0x55cd16e7ac00 2024-09-13T23:43:28.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1928088351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cd16a14340 con 0x55cd16e7ac00 2024-09-13T23:43:28.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee2b54d700 1 -- 172.21.15.167:0/1928088351 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55cd16e942c0 con 0x55cd16e7ac00 2024-09-13T23:43:28.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1928088351 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd16a15d40 con 0x55cd16e7ac00 2024-09-13T23:43:28.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee2b54d700 1 -- 172.21.15.167:0/1928088351 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55cd16afdb80 con 0x55cd16e7ac00 2024-09-13T23:43:28.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee2b54d700 1 --2- 172.21.15.167:0/1928088351 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16f06000 0x55cd16f0c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee2b54d700 1 -- 172.21.15.167:0/1928088351 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55cd16a69500 con 0x55cd16e7ac00 2024-09-13T23:43:28.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee47dcaf40 5 note: GC not initialized 2024-09-13T23:43:28.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.319+0000 7fee46d84700 1 --2- 172.21.15.167:0/1928088351 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16f06000 0x55cd16f0c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.323+0000 7fee46d84700 1 --2- 172.21.15.167:0/1928088351 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16f06000 0x55cd16f0c000 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55cd16e17cb0 tx=0x55cd16d677a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.323+0000 7fee28d48700 20 reqs_thread_entry: start 2024-09-13T23:43:28.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.323+0000 7fee47dcaf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:28.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.323+0000 7fee47dcaf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:28.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.323+0000 7fee26d44700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:58 2024 2024-09-13T23:43:28.332 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:28.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.323+0000 7fee26522700 20 reqs_thread_entry: start 2024-09-13T23:43:28.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.323+0000 7fee47dcaf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.323+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16e7a000 con 0x55cd16db1000 2024-09-13T23:43:28.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.323+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.323+0000 7fee47dcaf40 20 rados_obj.operate() r=0 bl.length=513 2024-09-13T23:43:28.333 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [ 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "foo:foo1", 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: "permissions": "full-control" 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:28.334 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [ 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo:foo1", 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:28.335 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:28.336 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:28.337 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:28.337 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:28.337 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:28.337 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.337 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:28.337 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:28.337 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:28.337 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:28.337 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:28.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.327+0000 7fee2d551700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:28.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.327+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94339340959744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16e7a400 con 0x55cd16db1000 2024-09-13T23:43:28.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.327+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94339340959744] v33'75 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.327+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94339339254528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4c800 con 0x55cd16db1400 2024-09-13T23:43:28.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.331+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94339339254528] v33'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.331+0000 7fee47dcaf40 20 remove_watcher() i=0 2024-09-13T23:43:28.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.331+0000 7fee47dcaf40 2 removed watcher, disabling cache 2024-09-13T23:43:28.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.331+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94339339255680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16d4d000 con 0x55cd16db1000 2024-09-13T23:43:28.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.331+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94339339255680] v33'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16db1000 2024-09-13T23:43:28.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.331+0000 7fee47dcaf40 20 remove_watcher() i=1 2024-09-13T23:43:28.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.331+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94339339256832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16db1800 con 0x55cd16db1400 2024-09-13T23:43:28.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.335+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94339339256832] v33'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.335+0000 7fee47dcaf40 20 remove_watcher() i=2 2024-09-13T23:43:28.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.335+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94339339257984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16f06400 con 0x55cd16d4c400 2024-09-13T23:43:28.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.339+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94339339257984] v33'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.339+0000 7fee47dcaf40 20 remove_watcher() i=3 2024-09-13T23:43:28.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.339+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94339339259136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16f06800 con 0x55cd16db1400 2024-09-13T23:43:28.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.343+0000 7fee46583700 1 -- 172.21.15.167:0/2613476649 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94339339259136] v33'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f200 con 0x55cd16db1400 2024-09-13T23:43:28.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.343+0000 7fee47dcaf40 20 remove_watcher() i=4 2024-09-13T23:43:28.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.343+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94339339260288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16f06c00 con 0x55cd16d4c000 2024-09-13T23:43:28.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.343+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94339339260288] v33'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.343+0000 7fee47dcaf40 20 remove_watcher() i=5 2024-09-13T23:43:28.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.343+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94339339261440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16f07000 con 0x55cd16d4c000 2024-09-13T23:43:28.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.347+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94339339261440] v33'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1f440 con 0x55cd16d4c000 2024-09-13T23:43:28.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.347+0000 7fee47dcaf40 20 remove_watcher() i=6 2024-09-13T23:43:28.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.347+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94339339262592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55cd16f07400 con 0x55cd16d4c400 2024-09-13T23:43:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.347+0000 7fee46d84700 1 -- 172.21.15.167:0/2613476649 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94339339262592] v33'62 uv60 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55cd16e1eb40 con 0x55cd16d4c400 2024-09-13T23:43:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.347+0000 7fee47dcaf40 20 remove_watcher() i=7 2024-09-13T23:43:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.347+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cd16db1400 msgr2=0x55cd16cea100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.347+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55cd16db1400 0x55cd16cea100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55cd16db1000 msgr2=0x55cd16ce9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55cd16db1000 0x55cd16ce9b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55cd16d4c400 msgr2=0x55cd16ceac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55cd16d4c400 0x55cd16ceac00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55cd16d4c000 msgr2=0x55cd16cea680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55cd16d4c000 0x55cd16cea680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16db0c00 msgr2=0x55cd16ce9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16db0c00 0x55cd16ce9600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x55cd16ce3920 tx=0x55cd16b07800 comp rx=0 tx=0).stop 2024-09-13T23:43:28.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 msgr2=0x55cd16ce8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee45d82700 1 -- 172.21.15.167:0/2613476649 reap_dead start 2024-09-13T23:43:28.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 0x55cd16ce8b00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55cd1787a3f0 tx=0x55cd16d66cc0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 shutdown_connections 2024-09-13T23:43:28.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/2613476649 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03800 0x55cd16ce8b00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 >> 172.21.15.167:0/2613476649 conn(0x55cd16a03400 msgr2=0x55cd16b03800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 shutdown_connections 2024-09-13T23:43:28.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/2613476649 wait complete. 2024-09-13T23:43:28.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1928088351 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16f06000 msgr2=0x55cd16f0c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/1928088351 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16f06000 0x55cd16f0c000 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x55cd16e17cb0 tx=0x55cd16d677a0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1928088351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 msgr2=0x55cd16ceb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/1928088351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 0x55cd16ceb180 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55cd16e92150 tx=0x55cd16e8a300 comp rx=0 tx=0).stop 2024-09-13T23:43:28.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1928088351 shutdown_connections 2024-09-13T23:43:28.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/1928088351 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16f06000 0x55cd16f0c000 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/1928088351 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16e7ac00 0x55cd16ceb180 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1928088351 >> 172.21.15.167:0/1928088351 conn(0x55cd16e7bc00 msgr2=0x55cd16b04000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1928088351 shutdown_connections 2024-09-13T23:43:28.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/1928088351 wait complete. 2024-09-13T23:43:28.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3063374043 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16db0000 msgr2=0x55cd16ce9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/3063374043 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16db0000 0x55cd16ce9080 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55cd1787b5c0 tx=0x55cd16d66240 comp rx=0 tx=0).stop 2024-09-13T23:43:28.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3063374043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 msgr2=0x55cd16ce8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/3063374043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 0x55cd16ce8580 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55cd16b5fec0 tx=0x55cd16b07a40 comp rx=0 tx=0).stop 2024-09-13T23:43:28.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3063374043 shutdown_connections 2024-09-13T23:43:28.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/3063374043 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55cd16db0000 0x55cd16ce9080 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 --2- 172.21.15.167:0/3063374043 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55cd16a03000 0x55cd16ce8580 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3063374043 >> 172.21.15.167:0/3063374043 conn(0x55cd16a02c00 msgr2=0x55cd16b03000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3063374043 shutdown_connections 2024-09-13T23:43:28.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.351+0000 7fee47dcaf40 1 -- 172.21.15.167:0/3063374043 wait complete. 2024-09-13T23:43:28.366 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:28.367 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-13T23:43:28.367 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-13T23:43:28.367 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-13T23:43:28.450 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:28.451 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:28.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d13746f40 1 Processor -- start 2024-09-13T23:43:28.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d13746f40 1 -- start start 2024-09-13T23:43:28.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d13746f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55e703221400 0x55e703506580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d13746f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e7032e5500 con 0x55e703221000 2024-09-13T23:43:28.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d13746f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e7032aea00 con 0x55e703221400 2024-09-13T23:43:28.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d116fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55e703221400 0x55e703506580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d11eff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55e703221000 0x55e703320800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45318/0 (socket says 172.21.15.167:45318) 2024-09-13T23:43:28.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d11eff700 1 -- 172.21.15.167:0/788901279 learned_addr learned my addr 172.21.15.167:0/788901279 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:28.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d116fe700 1 -- 172.21.15.167:0/788901279 >> v1:172.21.15.12:6789/0 conn(0x55e703221000 legacy=0x55e703320800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d116fe700 1 -- 172.21.15.167:0/788901279 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e703233d40 con 0x55e703221400 2024-09-13T23:43:28.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d116fe700 1 --2- 172.21.15.167:0/788901279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221400 0x55e703506580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55e703501920 tx=0x55e703325920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84b52e1b1331a18 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d10efd700 1 -- 172.21.15.167:0/788901279 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7032c54a0 con 0x55e703221400 2024-09-13T23:43:28.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d10efd700 1 -- 172.21.15.167:0/788901279 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e7032e5800 con 0x55e703221400 2024-09-13T23:43:28.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d10efd700 1 -- 172.21.15.167:0/788901279 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7032c5600 con 0x55e703221400 2024-09-13T23:43:28.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d13746f40 1 -- 172.21.15.167:0/788901279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221400 msgr2=0x55e703506580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d13746f40 1 --2- 172.21.15.167:0/788901279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221400 0x55e703506580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55e703501920 tx=0x55e703325920 comp rx=0 tx=0).stop 2024-09-13T23:43:28.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d13746f40 1 -- 172.21.15.167:0/788901279 shutdown_connections 2024-09-13T23:43:28.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d13746f40 1 --2- 172.21.15.167:0/788901279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221400 0x55e703506580 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d13746f40 1 -- 172.21.15.167:0/788901279 >> 172.21.15.167:0/788901279 conn(0x55e703220c00 msgr2=0x55e703320000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d13746f40 1 -- 172.21.15.167:0/788901279 shutdown_connections 2024-09-13T23:43:28.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.443+0000 7f3d13746f40 1 -- 172.21.15.167:0/788901279 wait complete. 2024-09-13T23:43:28.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.447+0000 7f3d13746f40 1 Processor -- start 2024-09-13T23:43:28.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.447+0000 7f3d13746f40 1 -- start start 2024-09-13T23:43:28.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.447+0000 7f3d13746f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 0x55e703506580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.447+0000 7f3d13746f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7032ae500 con 0x55e703221000 2024-09-13T23:43:28.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.447+0000 7f3d11eff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 0x55e703506580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.447+0000 7f3d11eff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 0x55e703506580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51222/0 (socket says 172.21.15.167:51222) 2024-09-13T23:43:28.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.447+0000 7f3d11eff700 1 -- 172.21.15.167:0/640854010 learned_addr learned my addr 172.21.15.167:0/640854010 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.447+0000 7f3d11eff700 1 -- 172.21.15.167:0/640854010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e703233ba0 con 0x55e703221000 2024-09-13T23:43:28.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.447+0000 7f3d11eff700 1 --2- 172.21.15.167:0/640854010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 0x55e703506580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55e703500b10 tx=0x55e703325b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510c94a25db37835 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.447+0000 7f3d106fc700 1 -- 172.21.15.167:0/640854010 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7032c54a0 con 0x55e703221000 2024-09-13T23:43:28.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.447+0000 7f3d106fc700 1 -- 172.21.15.167:0/640854010 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e7032e5b00 con 0x55e703221000 2024-09-13T23:43:28.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.447+0000 7f3d106fc700 1 -- 172.21.15.167:0/640854010 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7032c5600 con 0x55e703221000 2024-09-13T23:43:28.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d13746f40 1 -- 172.21.15.167:0/640854010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 msgr2=0x55e703506580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d13746f40 1 --2- 172.21.15.167:0/640854010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 0x55e703506580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55e703500b10 tx=0x55e703325b60 comp rx=0 tx=0).stop 2024-09-13T23:43:28.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d13746f40 1 -- 172.21.15.167:0/640854010 shutdown_connections 2024-09-13T23:43:28.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d13746f40 1 --2- 172.21.15.167:0/640854010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 0x55e703506580 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d13746f40 1 -- 172.21.15.167:0/640854010 >> 172.21.15.167:0/640854010 conn(0x55e703220c00 msgr2=0x55e703321000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d13746f40 1 -- 172.21.15.167:0/640854010 shutdown_connections 2024-09-13T23:43:28.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d13746f40 1 -- 172.21.15.167:0/640854010 wait complete. 2024-09-13T23:43:28.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d13746f40 1 Processor -- start 2024-09-13T23:43:28.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d13746f40 1 -- start start 2024-09-13T23:43:28.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d13746f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 0x55e703506580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d13746f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7032ae3c0 con 0x55e703221000 2024-09-13T23:43:28.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d11eff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 0x55e703506580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d11eff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 0x55e703506580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51228/0 (socket says 172.21.15.167:51228) 2024-09-13T23:43:28.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d11eff700 1 -- 172.21.15.167:0/370219982 learned_addr learned my addr 172.21.15.167:0/370219982 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d11eff700 1 -- 172.21.15.167:0/370219982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e703233a00 con 0x55e703221000 2024-09-13T23:43:28.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.451+0000 7f3d11eff700 1 --2- 172.21.15.167:0/370219982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 0x55e703506580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55e703500de0 tx=0x55e703325a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d0eef9700 1 -- 172.21.15.167:0/370219982 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7032c58c0 con 0x55e703221000 2024-09-13T23:43:28.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d0eef9700 1 -- 172.21.15.167:0/370219982 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e7032e5c80 con 0x55e703221000 2024-09-13T23:43:28.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d13746f40 1 -- 172.21.15.167:0/370219982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e703233860 con 0x55e703221000 2024-09-13T23:43:28.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d0eef9700 1 -- 172.21.15.167:0/370219982 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7032c54a0 con 0x55e703221000 2024-09-13T23:43:28.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d13746f40 1 -- 172.21.15.167:0/370219982 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7032336c0 con 0x55e703221000 2024-09-13T23:43:28.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d13746f40 1 Processor -- start 2024-09-13T23:43:28.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d13746f40 1 -- start start 2024-09-13T23:43:28.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d13746f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 0x55e703506b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d13746f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7032ae000 con 0x55e703221800 2024-09-13T23:43:28.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d12700700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 0x55e703506b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d0eef9700 1 -- 172.21.15.167:0/370219982 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e70331ab00 con 0x55e703221000 2024-09-13T23:43:28.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d12700700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 0x55e703506b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51232/0 (socket says 172.21.15.167:51232) 2024-09-13T23:43:28.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d12700700 1 -- 172.21.15.167:0/2729320208 learned_addr learned my addr 172.21.15.167:0/2729320208 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d0eef9700 1 --2- 172.21.15.167:0/370219982 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e7035d6000 0x55e703507080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d0eef9700 1 -- 172.21.15.167:0/370219982 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e703287180 con 0x55e703221000 2024-09-13T23:43:28.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d12700700 1 -- 172.21.15.167:0/2729320208 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e703232000 con 0x55e703221800 2024-09-13T23:43:28.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d11eff700 1 --2- 172.21.15.167:0/370219982 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e7035d6000 0x55e703507080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d12700700 1 --2- 172.21.15.167:0/2729320208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 0x55e703506b00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55e703557740 tx=0x55e703325f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89e1de9a706e2bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d0c6f4700 1 -- 172.21.15.167:0/2729320208 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7032c58c0 con 0x55e703221800 2024-09-13T23:43:28.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d0c6f4700 1 -- 172.21.15.167:0/2729320208 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e703576000 con 0x55e703221800 2024-09-13T23:43:28.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.455+0000 7f3d0c6f4700 1 -- 172.21.15.167:0/2729320208 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7032c54a0 con 0x55e703221800 2024-09-13T23:43:28.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 -- 172.21.15.167:0/2729320208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 msgr2=0x55e703506b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d11eff700 1 --2- 172.21.15.167:0/370219982 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e7035d6000 0x55e703507080 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55e7035575f0 tx=0x55e703588240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 --2- 172.21.15.167:0/2729320208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 0x55e703506b00 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55e703557740 tx=0x55e703325f20 comp rx=0 tx=0).stop 2024-09-13T23:43:28.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 -- 172.21.15.167:0/2729320208 shutdown_connections 2024-09-13T23:43:28.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 --2- 172.21.15.167:0/2729320208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 0x55e703506b00 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 -- 172.21.15.167:0/2729320208 >> 172.21.15.167:0/2729320208 conn(0x55e703221400 msgr2=0x55e703321800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 -- 172.21.15.167:0/2729320208 shutdown_connections 2024-09-13T23:43:28.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 -- 172.21.15.167:0/2729320208 wait complete. 2024-09-13T23:43:28.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 Processor -- start 2024-09-13T23:43:28.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 -- start start 2024-09-13T23:43:28.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 0x55e703506b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e703246640 con 0x55e703221800 2024-09-13T23:43:28.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d12700700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 0x55e703506b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d12700700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 0x55e703506b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51246/0 (socket says 172.21.15.167:51246) 2024-09-13T23:43:28.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 learned_addr learned my addr 172.21.15.167:0/3399900164 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e703232ea0 con 0x55e703221800 2024-09-13T23:43:28.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d12700700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 0x55e703506b00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55e7035561b0 tx=0x55e70358a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d0aef1700 1 -- 172.21.15.167:0/3399900164 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7032c5b80 con 0x55e703221800 2024-09-13T23:43:28.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d0aef1700 1 -- 172.21.15.167:0/3399900164 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e703576180 con 0x55e703221800 2024-09-13T23:43:28.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e703232820 con 0x55e703221800 2024-09-13T23:43:28.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d0aef1700 1 -- 172.21.15.167:0/3399900164 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7032c58c0 con 0x55e703221800 2024-09-13T23:43:28.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7032329c0 con 0x55e703221800 2024-09-13T23:43:28.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.459+0000 7f3d13746f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d0aef1700 1 -- 172.21.15.167:0/3399900164 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e70331adc0 con 0x55e703221800 2024-09-13T23:43:28.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d0aef1700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e7035d6c00 0x55e703507600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d0aef1700 1 -- 172.21.15.167:0/3399900164 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e703287340 con 0x55e703221800 2024-09-13T23:43:28.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d116fe700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e7035d6c00 0x55e703507600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e7035d7000 0x55e703507b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7400 con 0x55e7035d7000 2024-09-13T23:43:28.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d12700700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e7035d7000 0x55e703507b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d116fe700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e7035d6c00 0x55e703507600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55e703501920 tx=0x55e703325800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d12700700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e7035d7000 0x55e703507b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 20 realm 2024-09-13T23:43:28.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7000 2024-09-13T23:43:28.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:28.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7c00 con 0x55e7035d7000 2024-09-13T23:43:28.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e7035d7400 0x55e703508100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7400 2024-09-13T23:43:28.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.463+0000 7f3d11eff700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e7035d7400 0x55e703508100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d11eff700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e7035d7400 0x55e703508100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d13746f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:28.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d13746f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7c00 con 0x55e7035d7400 2024-09-13T23:43:28.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d13746f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:28.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d13746f40 20 searching for the correct realm 2024-09-13T23:43:28.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d13746f40 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e703542000 0x55e703508680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703542400 con 0x55e703542000 2024-09-13T23:43:28.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d116fe700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e703542000 0x55e703508680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d116fe700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e703542000 0x55e703508680 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.467+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703542400 con 0x55e7035d7400 2024-09-13T23:43:28.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d11eff700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e703542400 0x55e703508c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542400 2024-09-13T23:43:28.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d12700700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e703542400 0x55e703508c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d12700700 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e703542400 0x55e703508c00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7000 2024-09-13T23:43:28.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7000 2024-09-13T23:43:28.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542000 2024-09-13T23:43:28.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.471+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7000 2024-09-13T23:43:28.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7400 2024-09-13T23:43:28.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7400 2024-09-13T23:43:28.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7000 2024-09-13T23:43:28.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542400 2024-09-13T23:43:28.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7000 2024-09-13T23:43:28.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542400 2024-09-13T23:43:28.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.475+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7400 2024-09-13T23:43:28.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542000 2024-09-13T23:43:28.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7000 2024-09-13T23:43:28.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7400 2024-09-13T23:43:28.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542000 2024-09-13T23:43:28.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7000 2024-09-13T23:43:28.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7000 2024-09-13T23:43:28.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.479+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7000 2024-09-13T23:43:28.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.483+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.483+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542000 2024-09-13T23:43:28.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.483+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.483+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542400 2024-09-13T23:43:28.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.483+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.483+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7000 2024-09-13T23:43:28.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.483+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.483+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542400 2024-09-13T23:43:28.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.483+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.483+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542000 2024-09-13T23:43:28.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.483+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.483+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542400 2024-09-13T23:43:28.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7400 2024-09-13T23:43:28.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7400 2024-09-13T23:43:28.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542000 2024-09-13T23:43:28.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7400 2024-09-13T23:43:28.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7400 2024-09-13T23:43:28.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d13746f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:28.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d13746f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:28.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d13746f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:28.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d13746f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:28.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.487+0000 7f3d13746f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703542800 con 0x55e7035d7000 2024-09-13T23:43:28.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703542c00 con 0x55e7035d7400 2024-09-13T23:43:28.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:28.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703543000 con 0x55e7035d7400 2024-09-13T23:43:28.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:28.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 20 zone default found 2024-09-13T23:43:28.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 4 Realm: () 2024-09-13T23:43:28.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:28.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:28.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:28.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 20 zonegroup default 2024-09-13T23:43:28.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703543400 con 0x55e703542400 2024-09-13T23:43:28.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703543800 con 0x55e703542000 2024-09-13T23:43:28.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.491+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703543c00 con 0x55e703542000 2024-09-13T23:43:28.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.495+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.495+0000 7f3d13746f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.495+0000 7f3d13746f40 20 started sync module instance, tier type = 2024-09-13T23:43:28.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.495+0000 7f3d13746f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:28.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.495+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703543800 con 0x55e7035d7400 2024-09-13T23:43:28.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.495+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.495+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94450681371392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703543c00 con 0x55e7035d7400 2024-09-13T23:43:28.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.495+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703542c00 con 0x55e7035d7000 2024-09-13T23:43:28.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.499+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.499+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94450681372544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703543000 con 0x55e7035d7000 2024-09-13T23:43:28.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.499+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703543800 con 0x55e7035d7400 2024-09-13T23:43:28.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.499+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94450681371392] v33'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.503+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94450681372544] v33'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.503+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.503+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94450681373696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7c00 con 0x55e7035d7400 2024-09-13T23:43:28.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.503+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e703542400 2024-09-13T23:43:28.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.503+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.503+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94450681374848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703542800 con 0x55e703542400 2024-09-13T23:43:28.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.503+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703543400 con 0x55e7035d7400 2024-09-13T23:43:28.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.503+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94450681373696] v33'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.507+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94450681374848] v33'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.507+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.507+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94450681376000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703542c00 con 0x55e7035d7400 2024-09-13T23:43:28.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.507+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703543000 con 0x55e703542000 2024-09-13T23:43:28.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.511+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.511+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94450681377152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703696000 con 0x55e703542000 2024-09-13T23:43:28.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.511+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703696400 con 0x55e703542000 2024-09-13T23:43:28.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.511+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94450681376000] v33'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.511+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94450681377152] v33'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.511+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.511+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94450681378304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703696800 con 0x55e703542000 2024-09-13T23:43:28.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.511+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703696c00 con 0x55e703542400 2024-09-13T23:43:28.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'63 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94450681378304] v33'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94450681379456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703697000 con 0x55e703542400 2024-09-13T23:43:28.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d13746f40 20 add_watcher() i=0 2024-09-13T23:43:28.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d13746f40 20 add_watcher() i=1 2024-09-13T23:43:28.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d13746f40 20 add_watcher() i=2 2024-09-13T23:43:28.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d13746f40 20 add_watcher() i=3 2024-09-13T23:43:28.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d13746f40 20 add_watcher() i=4 2024-09-13T23:43:28.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d13746f40 20 add_watcher() i=5 2024-09-13T23:43:28.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d13746f40 20 add_watcher() i=6 2024-09-13T23:43:28.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94450681379456] v33'64 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d13746f40 20 add_watcher() i=7 2024-09-13T23:43:28.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d13746f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:28.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703697400 con 0x55e7035d7000 2024-09-13T23:43:28.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.515+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94450683068416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703697800 con 0x55e7035d7000 2024-09-13T23:43:28.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94450683068416] v33'76 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3cf8ecd700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:28.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3d13746f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:28.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3d13746f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:28.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3d13746f40 1 Processor -- start 2024-09-13T23:43:28.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3d13746f40 1 -- start start 2024-09-13T23:43:28.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3d13746f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 0x55e703509180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3d13746f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e703246780 con 0x55e703696c00 2024-09-13T23:43:28.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3d116fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 0x55e703509180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3d116fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 0x55e703509180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51252/0 (socket says 172.21.15.167:51252) 2024-09-13T23:43:28.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3d116fe700 1 -- 172.21.15.167:0/1681270199 learned_addr learned my addr 172.21.15.167:0/1681270199 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3d116fe700 1 -- 172.21.15.167:0/1681270199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e703232b60 con 0x55e703696c00 2024-09-13T23:43:28.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3d116fe700 1 --2- 172.21.15.167:0/1681270199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 0x55e703509180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55e7036a60f0 tx=0x55e703325860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8652d9713d139 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3cf86cc700 1 -- 172.21.15.167:0/1681270199 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7032c5760 con 0x55e703696c00 2024-09-13T23:43:28.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3cf86cc700 1 -- 172.21.15.167:0/1681270199 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e703576600 con 0x55e703696c00 2024-09-13T23:43:28.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.519+0000 7f3cf86cc700 1 -- 172.21.15.167:0/1681270199 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7036ae000 con 0x55e703696c00 2024-09-13T23:43:28.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d13746f40 1 -- 172.21.15.167:0/1681270199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 msgr2=0x55e703509180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d13746f40 1 --2- 172.21.15.167:0/1681270199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 0x55e703509180 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55e7036a60f0 tx=0x55e703325860 comp rx=0 tx=0).stop 2024-09-13T23:43:28.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d13746f40 1 -- 172.21.15.167:0/1681270199 shutdown_connections 2024-09-13T23:43:28.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d13746f40 1 --2- 172.21.15.167:0/1681270199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 0x55e703509180 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d13746f40 1 -- 172.21.15.167:0/1681270199 >> 172.21.15.167:0/1681270199 conn(0x55e703697c00 msgr2=0x55e703322000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d13746f40 1 -- 172.21.15.167:0/1681270199 shutdown_connections 2024-09-13T23:43:28.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d13746f40 1 -- 172.21.15.167:0/1681270199 wait complete. 2024-09-13T23:43:28.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d13746f40 1 Processor -- start 2024-09-13T23:43:28.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d13746f40 1 -- start start 2024-09-13T23:43:28.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d13746f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 0x55e703509180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d13746f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7032ae000 con 0x55e703696c00 2024-09-13T23:43:28.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d116fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 0x55e703509180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d116fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 0x55e703509180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51262/0 (socket says 172.21.15.167:51262) 2024-09-13T23:43:28.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d116fe700 1 -- 172.21.15.167:0/146103131 learned_addr learned my addr 172.21.15.167:0/146103131 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d116fe700 1 -- 172.21.15.167:0/146103131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e703232d00 con 0x55e703696c00 2024-09-13T23:43:28.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.523+0000 7f3d116fe700 1 --2- 172.21.15.167:0/146103131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 0x55e703509180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55e7036a6390 tx=0x55e703588000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.527+0000 7f3cf6ec9700 1 -- 172.21.15.167:0/146103131 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7036ae160 con 0x55e703696c00 2024-09-13T23:43:28.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.527+0000 7f3cf6ec9700 1 -- 172.21.15.167:0/146103131 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e703576300 con 0x55e703696c00 2024-09-13T23:43:28.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.527+0000 7f3d13746f40 1 -- 172.21.15.167:0/146103131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e703232340 con 0x55e703696c00 2024-09-13T23:43:28.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.527+0000 7f3cf6ec9700 1 -- 172.21.15.167:0/146103131 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e7036ae2c0 con 0x55e703696c00 2024-09-13T23:43:28.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.527+0000 7f3d13746f40 1 -- 172.21.15.167:0/146103131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e703233d40 con 0x55e703696c00 2024-09-13T23:43:28.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.531+0000 7f3cf6ec9700 1 -- 172.21.15.167:0/146103131 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e70331bb80 con 0x55e703696c00 2024-09-13T23:43:28.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.531+0000 7f3cf6ec9700 1 --2- 172.21.15.167:0/146103131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e70371e000 0x55e703724000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.531+0000 7f3cf6ec9700 1 -- 172.21.15.167:0/146103131 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e703287500 con 0x55e703696c00 2024-09-13T23:43:28.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.531+0000 7f3d12700700 1 --2- 172.21.15.167:0/146103131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e70371e000 0x55e703724000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.531+0000 7f3d13746f40 5 note: GC not initialized 2024-09-13T23:43:28.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.531+0000 7f3d12700700 1 --2- 172.21.15.167:0/146103131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e70371e000 0x55e703724000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55e703637500 tx=0x55e70358b6e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.531+0000 7f3cf46c4700 20 reqs_thread_entry: start 2024-09-13T23:43:28.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.535+0000 7f3d13746f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:28.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.535+0000 7f3d13746f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:28.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.535+0000 7f3cf26c0700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:58 2024 2024-09-13T23:43:28.543 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:28.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.535+0000 7f3cf1e9e700 20 reqs_thread_entry: start 2024-09-13T23:43:28.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.535+0000 7f3d13746f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.535+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703696000 con 0x55e7035d7000 2024-09-13T23:43:28.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.535+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.535+0000 7f3d13746f40 20 rados_obj.operate() r=0 bl.length=513 2024-09-13T23:43:28.544 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:28.544 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:28.544 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:28.544 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:28.544 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [ 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "foo:foo1", 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: "permissions": "full-control" 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:28.545 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [ 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo:foo1", 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:28.546 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.547 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:28.548 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:28.548 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:28.548 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:28.548 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:28.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.535+0000 7f3cf8ecd700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:28.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.535+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94450683068416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703696400 con 0x55e7035d7000 2024-09-13T23:43:28.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.539+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94450683068416] v33'77 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.539+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94450681371392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703542800 con 0x55e7035d7400 2024-09-13T23:43:28.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.543+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94450681371392] v33'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.543+0000 7f3d13746f40 20 remove_watcher() i=0 2024-09-13T23:43:28.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.543+0000 7f3d13746f40 2 removed watcher, disabling cache 2024-09-13T23:43:28.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.543+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94450681372544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e703543000 con 0x55e7035d7000 2024-09-13T23:43:28.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.543+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94450681372544] v33'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e7035d7000 2024-09-13T23:43:28.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.543+0000 7f3d13746f40 20 remove_watcher() i=1 2024-09-13T23:43:28.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.543+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94450681373696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e7035d7800 con 0x55e7035d7400 2024-09-13T23:43:28.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.547+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94450681373696] v33'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.547+0000 7f3d13746f40 20 remove_watcher() i=2 2024-09-13T23:43:28.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.547+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94450681374848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e70371e400 con 0x55e703542400 2024-09-13T23:43:28.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.551+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94450681374848] v33'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.551+0000 7f3d13746f40 20 remove_watcher() i=3 2024-09-13T23:43:28.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.551+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94450681376000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e70371e800 con 0x55e7035d7400 2024-09-13T23:43:28.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.551+0000 7f3d11eff700 1 -- 172.21.15.167:0/3399900164 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94450681376000] v33'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655200 con 0x55e7035d7400 2024-09-13T23:43:28.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.551+0000 7f3d13746f40 20 remove_watcher() i=4 2024-09-13T23:43:28.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.551+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94450681377152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e70371ec00 con 0x55e703542000 2024-09-13T23:43:28.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.555+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94450681377152] v33'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.555+0000 7f3d13746f40 20 remove_watcher() i=5 2024-09-13T23:43:28.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.555+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94450681378304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e70371f000 con 0x55e703542000 2024-09-13T23:43:28.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.555+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94450681378304] v33'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703655440 con 0x55e703542000 2024-09-13T23:43:28.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.555+0000 7f3d13746f40 20 remove_watcher() i=6 2024-09-13T23:43:28.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.555+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94450681379456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e70371f400 con 0x55e703542400 2024-09-13T23:43:28.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d12700700 1 -- 172.21.15.167:0/3399900164 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94450681379456] v33'65 uv63 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e703654b40 con 0x55e703542400 2024-09-13T23:43:28.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 20 remove_watcher() i=7 2024-09-13T23:43:28.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e7035d7400 msgr2=0x55e703508100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e7035d7400 0x55e703508100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e7035d7000 msgr2=0x55e703507b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e7035d7000 0x55e703507b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e703542400 msgr2=0x55e703508c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e703542400 0x55e703508c00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e703542000 msgr2=0x55e703508680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e703542000 0x55e703508680 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e7035d6c00 msgr2=0x55e703507600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e7035d6c00 0x55e703507600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x55e703501920 tx=0x55e703325800 comp rx=0 tx=0).stop 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 msgr2=0x55e703506b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d116fe700 1 -- 172.21.15.167:0/3399900164 reap_dead start 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 0x55e703506b00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55e7035561b0 tx=0x55e70358a480 comp rx=0 tx=0).stop 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 shutdown_connections 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 --2- 172.21.15.167:0/3399900164 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221800 0x55e703506b00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 >> 172.21.15.167:0/3399900164 conn(0x55e703221400 msgr2=0x55e703321800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 shutdown_connections 2024-09-13T23:43:28.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 -- 172.21.15.167:0/3399900164 wait complete. 2024-09-13T23:43:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 -- 172.21.15.167:0/146103131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e70371e000 msgr2=0x55e703724000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 --2- 172.21.15.167:0/146103131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e70371e000 0x55e703724000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55e703637500 tx=0x55e70358b6e0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 -- 172.21.15.167:0/146103131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 msgr2=0x55e703509180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.559+0000 7f3d13746f40 1 --2- 172.21.15.167:0/146103131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 0x55e703509180 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x55e7036a6390 tx=0x55e703588000 comp rx=0 tx=0).stop 2024-09-13T23:43:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 -- 172.21.15.167:0/146103131 shutdown_connections 2024-09-13T23:43:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 --2- 172.21.15.167:0/146103131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e70371e000 0x55e703724000 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 --2- 172.21.15.167:0/146103131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703696c00 0x55e703509180 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 -- 172.21.15.167:0/146103131 >> 172.21.15.167:0/146103131 conn(0x55e703697c00 msgr2=0x55e703322000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 -- 172.21.15.167:0/146103131 shutdown_connections 2024-09-13T23:43:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 -- 172.21.15.167:0/146103131 wait complete. 2024-09-13T23:43:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 -- 172.21.15.167:0/370219982 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e7035d6000 msgr2=0x55e703507080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 --2- 172.21.15.167:0/370219982 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e7035d6000 0x55e703507080 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55e7035575f0 tx=0x55e703588240 comp rx=0 tx=0).stop 2024-09-13T23:43:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 -- 172.21.15.167:0/370219982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 msgr2=0x55e703506580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 --2- 172.21.15.167:0/370219982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 0x55e703506580 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x55e703500de0 tx=0x55e703325a40 comp rx=0 tx=0).stop 2024-09-13T23:43:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 -- 172.21.15.167:0/370219982 shutdown_connections 2024-09-13T23:43:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 --2- 172.21.15.167:0/370219982 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e7035d6000 0x55e703507080 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 --2- 172.21.15.167:0/370219982 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e703221000 0x55e703506580 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 -- 172.21.15.167:0/370219982 >> 172.21.15.167:0/370219982 conn(0x55e703220c00 msgr2=0x55e703321000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 -- 172.21.15.167:0/370219982 shutdown_connections 2024-09-13T23:43:28.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.563+0000 7f3d13746f40 1 -- 172.21.15.167:0/370219982 wait complete. 2024-09-13T23:43:28.576 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:28.577 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-09-13T23:43:28.577 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-09-13T23:43:28.577 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo2 --secret ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy --key-type swift 2024-09-13T23:43:28.622 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:28.622 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc53b38bf40 1 Processor -- start 2024-09-13T23:43:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc53b38bf40 1 -- start start 2024-09-13T23:43:28.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc53b38bf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5629891c3000 0x5629894a8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc53b38bf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562989287500 con 0x5629891c3400 2024-09-13T23:43:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc53b38bf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x562989250a00 con 0x5629891c3000 2024-09-13T23:43:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc539b44700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5629891c3000 0x5629894a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc539b44700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5629891c3000 0x5629894a8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51274/0 (socket says 172.21.15.167:51274) 2024-09-13T23:43:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc539343700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5629891c3400 0x5629892c2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45320/0 (socket says 172.21.15.167:45320) 2024-09-13T23:43:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc539b44700 1 -- 172.21.15.167:0/2578527553 learned_addr learned my addr 172.21.15.167:0/2578527553 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc539b44700 1 -- 172.21.15.167:0/2578527553 >> v1:172.21.15.12:6789/0 conn(0x5629891c3400 legacy=0x5629892c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc539b44700 1 -- 172.21.15.167:0/2578527553 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629891d5d40 con 0x5629891c3000 2024-09-13T23:43:28.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc539b44700 1 --2- 172.21.15.167:0/2578527553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x5629894a3a70 tx=0x5629892c7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee940d26b9927c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc538b42700 1 -- 172.21.15.167:0/2578527553 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5629892674a0 con 0x5629891c3000 2024-09-13T23:43:28.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc538b42700 1 -- 172.21.15.167:0/2578527553 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562989287800 con 0x5629891c3000 2024-09-13T23:43:28.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc538b42700 1 -- 172.21.15.167:0/2578527553 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562989267600 con 0x5629891c3000 2024-09-13T23:43:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc53b38bf40 1 -- 172.21.15.167:0/2578527553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 msgr2=0x5629894a8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/2578527553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x5629894a3a70 tx=0x5629892c7920 comp rx=0 tx=0).stop 2024-09-13T23:43:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc53b38bf40 1 -- 172.21.15.167:0/2578527553 shutdown_connections 2024-09-13T23:43:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/2578527553 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc53b38bf40 1 -- 172.21.15.167:0/2578527553 >> 172.21.15.167:0/2578527553 conn(0x5629891c2c00 msgr2=0x5629892c2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc53b38bf40 1 -- 172.21.15.167:0/2578527553 shutdown_connections 2024-09-13T23:43:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.615+0000 7fc53b38bf40 1 -- 172.21.15.167:0/2578527553 wait complete. 2024-09-13T23:43:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 Processor -- start 2024-09-13T23:43:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 -- start start 2024-09-13T23:43:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562989250500 con 0x5629891c3000 2024-09-13T23:43:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc539b44700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc539b44700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51276/0 (socket says 172.21.15.167:51276) 2024-09-13T23:43:28.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc539b44700 1 -- 172.21.15.167:0/2769904877 learned_addr learned my addr 172.21.15.167:0/2769904877 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc539b44700 1 -- 172.21.15.167:0/2769904877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629891d5ba0 con 0x5629891c3000 2024-09-13T23:43:28.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc539b44700 1 --2- 172.21.15.167:0/2769904877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x56298931e1b0 tx=0x5629892c7800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bfe9d77c269fac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc538341700 1 -- 172.21.15.167:0/2769904877 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562989267760 con 0x5629891c3000 2024-09-13T23:43:28.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc538341700 1 -- 172.21.15.167:0/2769904877 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562989287b00 con 0x5629891c3000 2024-09-13T23:43:28.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc538341700 1 -- 172.21.15.167:0/2769904877 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5629892674a0 con 0x5629891c3000 2024-09-13T23:43:28.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 -- 172.21.15.167:0/2769904877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 msgr2=0x5629894a8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/2769904877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x56298931e1b0 tx=0x5629892c7800 comp rx=0 tx=0).stop 2024-09-13T23:43:28.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 -- 172.21.15.167:0/2769904877 shutdown_connections 2024-09-13T23:43:28.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/2769904877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 -- 172.21.15.167:0/2769904877 >> 172.21.15.167:0/2769904877 conn(0x5629891c2c00 msgr2=0x5629892c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 -- 172.21.15.167:0/2769904877 shutdown_connections 2024-09-13T23:43:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 -- 172.21.15.167:0/2769904877 wait complete. 2024-09-13T23:43:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 Processor -- start 2024-09-13T23:43:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 -- start start 2024-09-13T23:43:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc53b38bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629892503c0 con 0x5629891c3000 2024-09-13T23:43:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc539b44700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc539b44700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51280/0 (socket says 172.21.15.167:51280) 2024-09-13T23:43:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.619+0000 7fc539b44700 1 -- 172.21.15.167:0/1621442634 learned_addr learned my addr 172.21.15.167:0/1621442634 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc539b44700 1 -- 172.21.15.167:0/1621442634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629891d5a00 con 0x5629891c3000 2024-09-13T23:43:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc539b44700 1 --2- 172.21.15.167:0/1621442634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x56298931ff80 tx=0x5629892c7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc536b3e700 1 -- 172.21.15.167:0/1621442634 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562989267600 con 0x5629891c3000 2024-09-13T23:43:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc536b3e700 1 -- 172.21.15.167:0/1621442634 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562989287c80 con 0x5629891c3000 2024-09-13T23:43:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621442634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5629891d5860 con 0x5629891c3000 2024-09-13T23:43:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc536b3e700 1 -- 172.21.15.167:0/1621442634 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5629892678c0 con 0x5629891c3000 2024-09-13T23:43:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621442634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629891d56c0 con 0x5629891c3000 2024-09-13T23:43:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc53b38bf40 1 Processor -- start 2024-09-13T23:43:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc53b38bf40 1 -- start start 2024-09-13T23:43:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc53b38bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 0x5629894a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc53b38bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629891e8640 con 0x5629891c3800 2024-09-13T23:43:28.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc53a345700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 0x5629894a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc53a345700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 0x5629894a8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51288/0 (socket says 172.21.15.167:51288) 2024-09-13T23:43:28.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc53a345700 1 -- 172.21.15.167:0/3377293404 learned_addr learned my addr 172.21.15.167:0/3377293404 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc536b3e700 1 -- 172.21.15.167:0/1621442634 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5629892bcb00 con 0x5629891c3000 2024-09-13T23:43:28.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc53a345700 1 -- 172.21.15.167:0/3377293404 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629891d4000 con 0x5629891c3800 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc536b3e700 1 --2- 172.21.15.167:0/1621442634 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56298956e000 0x5629894a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc53a345700 1 --2- 172.21.15.167:0/3377293404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 0x5629894a8b00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5629894ed290 tx=0x56298952a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0c5d576f5bbdf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc536b3e700 1 -- 172.21.15.167:0/1621442634 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x562989229180 con 0x5629891c3000 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc539b44700 1 --2- 172.21.15.167:0/1621442634 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56298956e000 0x5629894a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.623+0000 7fc534339700 1 -- 172.21.15.167:0/3377293404 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562989267600 con 0x5629891c3800 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc534339700 1 -- 172.21.15.167:0/3377293404 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562989524000 con 0x5629891c3800 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc534339700 1 -- 172.21.15.167:0/3377293404 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5629892678c0 con 0x5629891c3800 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3377293404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 msgr2=0x5629894a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/3377293404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 0x5629894a8b00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x5629894ed290 tx=0x56298952a8a0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3377293404 shutdown_connections 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/3377293404 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 0x5629894a8b00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3377293404 >> 172.21.15.167:0/3377293404 conn(0x5629891c3400 msgr2=0x5629892c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3377293404 shutdown_connections 2024-09-13T23:43:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc539b44700 1 --2- 172.21.15.167:0/1621442634 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56298956e000 0x5629894a9080 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x5629894ed680 tx=0x56298952a120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3377293404 wait complete. 2024-09-13T23:43:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 Processor -- start 2024-09-13T23:43:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 -- start start 2024-09-13T23:43:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 0x5629894a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629891e88c0 con 0x5629891c3800 2024-09-13T23:43:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53a345700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 0x5629894a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53a345700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 0x5629894a8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51296/0 (socket says 172.21.15.167:51296) 2024-09-13T23:43:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 learned_addr learned my addr 172.21.15.167:0/3027446338 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629891d4ea0 con 0x5629891c3800 2024-09-13T23:43:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53a345700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 0x5629894a8b00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5629894ed080 tx=0x5629892c7b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc532b36700 1 -- 172.21.15.167:0/3027446338 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562989267760 con 0x5629891c3800 2024-09-13T23:43:28.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc532b36700 1 -- 172.21.15.167:0/3027446338 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562989524180 con 0x5629891c3800 2024-09-13T23:43:28.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc532b36700 1 -- 172.21.15.167:0/3027446338 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562989267600 con 0x5629891c3800 2024-09-13T23:43:28.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5629891d4820 con 0x5629891c3800 2024-09-13T23:43:28.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.627+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629891d49c0 con 0x5629891c3800 2024-09-13T23:43:28.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc532b36700 1 -- 172.21.15.167:0/3027446338 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5629892bcdc0 con 0x5629891c3800 2024-09-13T23:43:28.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc532b36700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56298956ec00 0x5629894a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc532b36700 1 -- 172.21.15.167:0/3027446338 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x562989229340 con 0x5629891c3800 2024-09-13T23:43:28.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc539343700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56298956ec00 0x5629894a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56298956f000 0x5629894a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f400 con 0x56298956f000 2024-09-13T23:43:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc539343700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56298956ec00 0x5629894a9600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5629894a3f80 tx=0x56298952bf80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53a345700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56298956f000 0x5629894a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53a345700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56298956f000 0x5629894a9b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 20 realm 2024-09-13T23:43:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f000 2024-09-13T23:43:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956fc00 con 0x56298956f000 2024-09-13T23:43:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56298956f400 0x5629894aa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.631+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f400 2024-09-13T23:43:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc539b44700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56298956f400 0x5629894aa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc539b44700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56298956f400 0x5629894aa100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc53b38bf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:28.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956fc00 con 0x56298956f400 2024-09-13T23:43:28.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc53b38bf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:28.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc53b38bf40 20 searching for the correct realm 2024-09-13T23:43:28.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56298962c000 0x5629894aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962c400 con 0x56298962c000 2024-09-13T23:43:28.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc539343700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56298962c000 0x5629894aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.635+0000 7fc539343700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56298962c000 0x5629894aa680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962c400 con 0x56298956f400 2024-09-13T23:43:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc539b44700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56298962c400 0x5629894aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c400 2024-09-13T23:43:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc53a345700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56298962c400 0x5629894aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc53a345700 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56298962c400 0x5629894aac00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f000 2024-09-13T23:43:28.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f000 2024-09-13T23:43:28.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.639+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c000 2024-09-13T23:43:28.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f000 2024-09-13T23:43:28.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f400 2024-09-13T23:43:28.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f400 2024-09-13T23:43:28.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f000 2024-09-13T23:43:28.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c400 2024-09-13T23:43:28.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f000 2024-09-13T23:43:28.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.643+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c400 2024-09-13T23:43:28.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.647+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.647+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f400 2024-09-13T23:43:28.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.647+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.647+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c000 2024-09-13T23:43:28.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.647+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.647+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f000 2024-09-13T23:43:28.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.647+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.647+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f400 2024-09-13T23:43:28.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.647+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.647+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c000 2024-09-13T23:43:28.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f000 2024-09-13T23:43:28.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f000 2024-09-13T23:43:28.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f000 2024-09-13T23:43:28.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c000 2024-09-13T23:43:28.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c400 2024-09-13T23:43:28.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f000 2024-09-13T23:43:28.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c400 2024-09-13T23:43:28.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.651+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c000 2024-09-13T23:43:28.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.655+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.655+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c400 2024-09-13T23:43:28.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.655+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.655+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f400 2024-09-13T23:43:28.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.655+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.655+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f400 2024-09-13T23:43:28.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.655+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.655+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c000 2024-09-13T23:43:28.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.655+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.655+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f400 2024-09-13T23:43:28.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f400 2024-09-13T23:43:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962c800 con 0x56298956f000 2024-09-13T23:43:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962cc00 con 0x56298956f400 2024-09-13T23:43:28.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:28.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962d000 con 0x56298956f400 2024-09-13T23:43:28.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:28.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 zone default found 2024-09-13T23:43:28.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 4 Realm: () 2024-09-13T23:43:28.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:28.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:28.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:28.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 zonegroup default 2024-09-13T23:43:28.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.659+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962d400 con 0x56298962c400 2024-09-13T23:43:28.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc53b38bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962d800 con 0x56298962c000 2024-09-13T23:43:28.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc53b38bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962dc00 con 0x56298962c000 2024-09-13T23:43:28.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc53b38bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc53b38bf40 20 started sync module instance, tier type = 2024-09-13T23:43:28.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc53b38bf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:28.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.663+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962d800 con 0x56298956f400 2024-09-13T23:43:28.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.667+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.667+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94736396974848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962dc00 con 0x56298956f400 2024-09-13T23:43:28.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.667+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962cc00 con 0x56298956f000 2024-09-13T23:43:28.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.667+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.667+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94736396974848] v33'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.667+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94736396976000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962d000 con 0x56298956f000 2024-09-13T23:43:28.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.667+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962d800 con 0x56298956f400 2024-09-13T23:43:28.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.671+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94736396976000] v33'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.671+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.671+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94736396977152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956fc00 con 0x56298956f400 2024-09-13T23:43:28.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.671+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298962c400 2024-09-13T23:43:28.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.675+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.675+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94736396978304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962c800 con 0x56298962c400 2024-09-13T23:43:28.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.675+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962d400 con 0x56298956f400 2024-09-13T23:43:28.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.675+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94736396977152] v33'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.675+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94736396978304] v33'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.675+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.675+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94736396979456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962cc00 con 0x56298956f400 2024-09-13T23:43:28.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.679+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962d000 con 0x56298962c000 2024-09-13T23:43:28.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.679+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.679+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94736396980608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962e000 con 0x56298962c000 2024-09-13T23:43:28.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.679+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962e400 con 0x56298962c000 2024-09-13T23:43:28.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.679+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94736396979456] v33'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.679+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.679+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94736396981760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962e800 con 0x56298962c000 2024-09-13T23:43:28.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.679+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962ec00 con 0x56298962c400 2024-09-13T23:43:28.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94736396980608] v33'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'66 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94736396981760] v33'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94736396982912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962f000 con 0x56298962c400 2024-09-13T23:43:28.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53b38bf40 20 add_watcher() i=0 2024-09-13T23:43:28.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53b38bf40 20 add_watcher() i=1 2024-09-13T23:43:28.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53b38bf40 20 add_watcher() i=2 2024-09-13T23:43:28.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53b38bf40 20 add_watcher() i=3 2024-09-13T23:43:28.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53b38bf40 20 add_watcher() i=4 2024-09-13T23:43:28.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53b38bf40 20 add_watcher() i=5 2024-09-13T23:43:28.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53b38bf40 20 add_watcher() i=6 2024-09-13T23:43:28.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94736396982912] v33'67 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53b38bf40 20 add_watcher() i=7 2024-09-13T23:43:28.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53b38bf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:28.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.683+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962f400 con 0x56298956f000 2024-09-13T23:43:28.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94736398630912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962f800 con 0x56298956f000 2024-09-13T23:43:28.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94736398630912] v33'78 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc520b12700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc53b38bf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc53b38bf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc53b38bf40 1 Processor -- start 2024-09-13T23:43:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc53b38bf40 1 -- start start 2024-09-13T23:43:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc53b38bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 0x5629894ab180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc53b38bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629892503c0 con 0x56298962ec00 2024-09-13T23:43:28.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc539343700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 0x5629894ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc539343700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 0x5629894ab180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51306/0 (socket says 172.21.15.167:51306) 2024-09-13T23:43:28.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.687+0000 7fc539343700 1 -- 172.21.15.167:0/159319568 learned_addr learned my addr 172.21.15.167:0/159319568 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc539343700 1 -- 172.21.15.167:0/159319568 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629891d4b60 con 0x56298962ec00 2024-09-13T23:43:28.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc539343700 1 --2- 172.21.15.167:0/159319568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 0x5629894ab180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x56298963e060 tx=0x562989640000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a553bee3a53dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc520311700 1 -- 172.21.15.167:0/159319568 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562989648000 con 0x56298962ec00 2024-09-13T23:43:28.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc520311700 1 -- 172.21.15.167:0/159319568 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562989524600 con 0x56298962ec00 2024-09-13T23:43:28.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc520311700 1 -- 172.21.15.167:0/159319568 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562989648160 con 0x56298962ec00 2024-09-13T23:43:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc53b38bf40 1 -- 172.21.15.167:0/159319568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 msgr2=0x5629894ab180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/159319568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 0x5629894ab180 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x56298963e060 tx=0x562989640000 comp rx=0 tx=0).stop 2024-09-13T23:43:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc53b38bf40 1 -- 172.21.15.167:0/159319568 shutdown_connections 2024-09-13T23:43:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/159319568 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 0x5629894ab180 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc53b38bf40 1 -- 172.21.15.167:0/159319568 >> 172.21.15.167:0/159319568 conn(0x56298962fc00 msgr2=0x5629892c4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc53b38bf40 1 -- 172.21.15.167:0/159319568 shutdown_connections 2024-09-13T23:43:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc53b38bf40 1 -- 172.21.15.167:0/159319568 wait complete. 2024-09-13T23:43:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc53b38bf40 1 Processor -- start 2024-09-13T23:43:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc53b38bf40 1 -- start start 2024-09-13T23:43:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc53b38bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 0x5629894ab180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc53b38bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562989250500 con 0x56298962ec00 2024-09-13T23:43:28.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc539343700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 0x5629894ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc539343700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 0x5629894ab180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51318/0 (socket says 172.21.15.167:51318) 2024-09-13T23:43:28.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc539343700 1 -- 172.21.15.167:0/1621367742 learned_addr learned my addr 172.21.15.167:0/1621367742 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.691+0000 7fc539343700 1 -- 172.21.15.167:0/1621367742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629891d4d00 con 0x56298962ec00 2024-09-13T23:43:28.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.695+0000 7fc539343700 1 --2- 172.21.15.167:0/1621367742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 0x5629894ab180 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x56298963f320 tx=0x5629896402a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.695+0000 7fc51eb0e700 1 -- 172.21.15.167:0/1621367742 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5629896482c0 con 0x56298962ec00 2024-09-13T23:43:28.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.695+0000 7fc51eb0e700 1 -- 172.21.15.167:0/1621367742 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562989524300 con 0x56298962ec00 2024-09-13T23:43:28.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.695+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621367742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5629891d4340 con 0x56298962ec00 2024-09-13T23:43:28.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.695+0000 7fc51eb0e700 1 -- 172.21.15.167:0/1621367742 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562989648000 con 0x56298962ec00 2024-09-13T23:43:28.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.695+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621367742 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629891d4b60 con 0x56298962ec00 2024-09-13T23:43:28.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.695+0000 7fc51eb0e700 1 -- 172.21.15.167:0/1621367742 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5629892bdb80 con 0x56298962ec00 2024-09-13T23:43:28.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.695+0000 7fc51eb0e700 1 --2- 172.21.15.167:0/1621367742 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5629896c2000 0x5629896c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.695+0000 7fc51eb0e700 1 -- 172.21.15.167:0/1621367742 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x562989229500 con 0x56298962ec00 2024-09-13T23:43:28.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.695+0000 7fc53b38bf40 5 note: GC not initialized 2024-09-13T23:43:28.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.695+0000 7fc53a345700 1 --2- 172.21.15.167:0/1621367742 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5629896c2000 0x5629896c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.695+0000 7fc53a345700 1 --2- 172.21.15.167:0/1621367742 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5629896c2000 0x5629896c8000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5629895c1d70 tx=0x56298952b3e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc51c309700 20 reqs_thread_entry: start 2024-09-13T23:43:28.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:28.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:28.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc51a305700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:59 2024 2024-09-13T23:43:28.707 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:28.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:28.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962e000 con 0x56298956f000 2024-09-13T23:43:28.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc519ae3700 20 reqs_thread_entry: start 2024-09-13T23:43:28.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv7 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 20 rados_obj.operate() r=0 bl.length=513 2024-09-13T23:43:28.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:28.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:28.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:28.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : miss 2024-09-13T23:43:28.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.2 8:45671577:users.swift::foo%3afoo2:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962e400 con 0x56298962c400 2024-09-13T23:43:28.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 foo:foo2 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 194+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x0 2024-09-13T23:43:28.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 10 adding default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-09-13T23:43:28.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : hit (negative entry) 2024-09-13T23:43:28.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.699+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~621 in=621b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962d000 con 0x56298956f000 2024-09-13T23:43:28.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~621] v33'8 uv8 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc53b38bf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:28.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc53b38bf40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:28.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc53b38bf40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:28.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94736398633216 in=809b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956f800 con 0x56298956f400 2024-09-13T23:43:28.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94736396979456 notify 141733920778 ret 0) v3 ==== 839+0+0 (crc 0 0 0) 0x562989528680 con 0x56298956f400 2024-09-13T23:43:28.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc534339700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920778 cookie 94736396979456 notifier 4406 bl.length()=797 2024-09-13T23:43:28.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 notify.4 [notify cookie 94736398633216 out=8b] v0'0 uv66 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc534339700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:28.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc534339700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:28.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc534339700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5629896c2400 con 0x56298956f400 2024-09-13T23:43:28.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94736398633216 notify 141733920778 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562989528820 con 0x56298956f400 2024-09-13T23:43:28.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.703+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 8.2 8:45671577:users.swift::foo%3afoo2:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5629896c2c00 con 0x56298962c400 2024-09-13T23:43:28.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(67 foo:foo2 [delete,create,writefull 0~7] v33'1 uv1 ondisk = 0) v8 ==== 236+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc53b38bf40 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-09-13T23:43:28.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc53b38bf40 10 moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-09-13T23:43:28.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc53b38bf40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-09-13T23:43:28.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94736398633216 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5629896c3000 con 0x56298956f400 2024-09-13T23:43:28.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== watch-notify(notify (1) cookie 94736396974848 notify 141733920779 ret 0) v3 ==== 208+0+0 (crc 0 0 0) 0x562989528820 con 0x56298956f400 2024-09-13T23:43:28.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc534339700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920779 cookie 94736396974848 notifier 4406 bl.length()=166 2024-09-13T23:43:28.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(68 notify.0 [notify cookie 94736398633216 out=8b] v0'0 uv66 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc534339700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-09-13T23:43:28.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc534339700 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-09-13T23:43:28.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc534339700 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5629896c2800 con 0x56298956f400 2024-09-13T23:43:28.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.707+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== watch-notify(notify_complete (2) cookie 94736398633216 notify 141733920779 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5629895289c0 con 0x56298956f400 2024-09-13T23:43:28.718 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:28.718 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:28.718 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:28.718 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:28.718 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:28.718 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:28.718 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [ 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "foo:foo1", 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: "permissions": "full-control" 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "foo:foo2", 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: "permissions": "" 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:28.719 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [ 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo:foo1", 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo:foo2", 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.720 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.721 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:28.722 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:28.722 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:28.722 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:28.722 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:28.722 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:28.722 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.722 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:28.722 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:28.722 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:28.722 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:28.722 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:28.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.711+0000 7fc520b12700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:28.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.711+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94736398630912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5629896c3800 con 0x56298956f000 2024-09-13T23:43:28.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.711+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94736398630912] v33'79 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.711+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94736396974848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5629896c3c00 con 0x56298956f400 2024-09-13T23:43:28.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.715+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94736396974848] v33'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.715+0000 7fc53b38bf40 20 remove_watcher() i=0 2024-09-13T23:43:28.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.715+0000 7fc53b38bf40 2 removed watcher, disabling cache 2024-09-13T23:43:28.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.715+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94736396976000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298956fc00 con 0x56298956f000 2024-09-13T23:43:28.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.715+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94736396976000] v33'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298956f000 2024-09-13T23:43:28.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.715+0000 7fc53b38bf40 20 remove_watcher() i=1 2024-09-13T23:43:28.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.715+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94736396977152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5629896c3400 con 0x56298956f400 2024-09-13T23:43:28.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.719+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94736396977152] v33'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.719+0000 7fc53b38bf40 20 remove_watcher() i=2 2024-09-13T23:43:28.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.719+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94736396978304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5629896c2400 con 0x56298962c400 2024-09-13T23:43:28.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.723+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94736396978304] v33'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.723+0000 7fc53b38bf40 20 remove_watcher() i=3 2024-09-13T23:43:28.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.723+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94736396979456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5629896c3000 con 0x56298956f400 2024-09-13T23:43:28.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.723+0000 7fc539b44700 1 -- 172.21.15.167:0/3027446338 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94736396979456] v33'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621200 con 0x56298956f400 2024-09-13T23:43:28.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.723+0000 7fc53b38bf40 20 remove_watcher() i=4 2024-09-13T23:43:28.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.723+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94736396980608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5629896c2800 con 0x56298962c000 2024-09-13T23:43:28.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.727+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94736396980608] v33'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.727+0000 7fc53b38bf40 20 remove_watcher() i=5 2024-09-13T23:43:28.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.727+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94736396981760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962cc00 con 0x56298962c000 2024-09-13T23:43:28.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.727+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94736396981760] v33'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989621440 con 0x56298962c000 2024-09-13T23:43:28.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 20 remove_watcher() i=6 2024-09-13T23:43:28.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94736396982912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56298962d400 con 0x56298962c400 2024-09-13T23:43:28.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53a345700 1 -- 172.21.15.167:0/3027446338 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94736396982912] v33'68 uv66 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562989620b40 con 0x56298962c400 2024-09-13T23:43:28.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 20 remove_watcher() i=7 2024-09-13T23:43:28.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56298956f400 msgr2=0x5629894aa100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56298956f400 0x5629894aa100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56298956f000 msgr2=0x5629894a9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56298956f000 0x5629894a9b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56298962c400 msgr2=0x5629894aac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56298962c400 0x5629894aac00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56298962c000 msgr2=0x5629894aa680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56298962c000 0x5629894aa680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56298956ec00 msgr2=0x5629894a9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56298956ec00 0x5629894a9600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x5629894a3f80 tx=0x56298952bf80 comp rx=0 tx=0).stop 2024-09-13T23:43:28.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 msgr2=0x5629894a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 0x5629894a8b00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x5629894ed080 tx=0x5629892c7b60 comp rx=0 tx=0).stop 2024-09-13T23:43:28.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc539343700 1 -- 172.21.15.167:0/3027446338 reap_dead start 2024-09-13T23:43:28.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 shutdown_connections 2024-09-13T23:43:28.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/3027446338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3800 0x5629894a8b00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 >> 172.21.15.167:0/3027446338 conn(0x5629891c3400 msgr2=0x5629892c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 shutdown_connections 2024-09-13T23:43:28.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.731+0000 7fc53b38bf40 1 -- 172.21.15.167:0/3027446338 wait complete. 2024-09-13T23:43:28.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621367742 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5629896c2000 msgr2=0x5629896c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/1621367742 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5629896c2000 0x5629896c8000 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x5629895c1d70 tx=0x56298952b3e0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621367742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 msgr2=0x5629894ab180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/1621367742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 0x5629894ab180 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x56298963f320 tx=0x5629896402a0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621367742 shutdown_connections 2024-09-13T23:43:28.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/1621367742 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5629896c2000 0x5629896c8000 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/1621367742 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56298962ec00 0x5629894ab180 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621367742 >> 172.21.15.167:0/1621367742 conn(0x56298962fc00 msgr2=0x5629892c4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621367742 shutdown_connections 2024-09-13T23:43:28.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621367742 wait complete. 2024-09-13T23:43:28.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621442634 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56298956e000 msgr2=0x5629894a9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/1621442634 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56298956e000 0x5629894a9080 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x5629894ed680 tx=0x56298952a120 comp rx=0 tx=0).stop 2024-09-13T23:43:28.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621442634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 msgr2=0x5629894a8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/1621442634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x56298931ff80 tx=0x5629892c7920 comp rx=0 tx=0).stop 2024-09-13T23:43:28.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621442634 shutdown_connections 2024-09-13T23:43:28.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/1621442634 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56298956e000 0x5629894a9080 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 --2- 172.21.15.167:0/1621442634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5629891c3000 0x5629894a8580 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621442634 >> 172.21.15.167:0/1621442634 conn(0x5629891c2c00 msgr2=0x5629892c3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621442634 shutdown_connections 2024-09-13T23:43:28.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.735+0000 7fc53b38bf40 1 -- 172.21.15.167:0/1621442634 wait complete. 2024-09-13T23:43:28.749 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:28.749 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-13T23:43:28.749 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-13T23:43:28.750 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-13T23:43:28.790 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:28.790 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:28.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 Processor -- start 2024-09-13T23:43:28.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 -- start start 2024-09-13T23:43:28.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55aa0e855400 0x55aa0eb3a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55aa0e919500 con 0x55aa0e855000 2024-09-13T23:43:28.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55aa0e8e2a00 con 0x55aa0e855400 2024-09-13T23:43:28.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51bf57c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55aa0e855400 0x55aa0eb3a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51bf57c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55aa0e855400 0x55aa0eb3a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51332/0 (socket says 172.21.15.167:51332) 2024-09-13T23:43:28.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51bf57c700 1 -- 172.21.15.167:0/2620446134 learned_addr learned my addr 172.21.15.167:0/2620446134 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51bfd7d700 1 --1- 172.21.15.167:0/2620446134 >> v1:172.21.15.12:6789/0 conn(0x55aa0e855000 0x55aa0e954800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45332/0 (socket says 172.21.15.167:45332) 2024-09-13T23:43:28.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51bf57c700 1 -- 172.21.15.167:0/2620446134 >> v1:172.21.15.12:6789/0 conn(0x55aa0e855000 legacy=0x55aa0e954800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51bf57c700 1 -- 172.21.15.167:0/2620446134 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa0e867d40 con 0x55aa0e855400 2024-09-13T23:43:28.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51bf57c700 1 --2- 172.21.15.167:0/2620446134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855400 0x55aa0eb3a580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55aa0eb35920 tx=0x55aa0e959920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a4721c93ef509a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51bed7b700 1 -- 172.21.15.167:0/2620446134 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0e8f94a0 con 0x55aa0e855400 2024-09-13T23:43:28.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51bed7b700 1 -- 172.21.15.167:0/2620446134 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa0e919800 con 0x55aa0e855400 2024-09-13T23:43:28.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51bed7b700 1 -- 172.21.15.167:0/2620446134 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0e8f9600 con 0x55aa0e855400 2024-09-13T23:43:28.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2620446134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855400 msgr2=0x55aa0eb3a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2620446134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855400 0x55aa0eb3a580 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x55aa0eb35920 tx=0x55aa0e959920 comp rx=0 tx=0).stop 2024-09-13T23:43:28.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2620446134 shutdown_connections 2024-09-13T23:43:28.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2620446134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855400 0x55aa0eb3a580 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2620446134 >> 172.21.15.167:0/2620446134 conn(0x55aa0e854c00 msgr2=0x55aa0e954000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2620446134 shutdown_connections 2024-09-13T23:43:28.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2620446134 wait complete. 2024-09-13T23:43:28.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 Processor -- start 2024-09-13T23:43:28.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 -- start start 2024-09-13T23:43:28.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 0x55aa0eb3a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.783+0000 7f51c15c4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0e8e2500 con 0x55aa0e855000 2024-09-13T23:43:28.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51bfd7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 0x55aa0eb3a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51bfd7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 0x55aa0eb3a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51340/0 (socket says 172.21.15.167:51340) 2024-09-13T23:43:28.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2237075342 learned_addr learned my addr 172.21.15.167:0/2237075342 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2237075342 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa0e867ba0 con 0x55aa0e855000 2024-09-13T23:43:28.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51bfd7d700 1 --2- 172.21.15.167:0/2237075342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 0x55aa0eb3a580 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55aa0e9b1e90 tx=0x55aa0e959b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99214474c9a81170 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51be57a700 1 -- 172.21.15.167:0/2237075342 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0e8f94a0 con 0x55aa0e855000 2024-09-13T23:43:28.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51be57a700 1 -- 172.21.15.167:0/2237075342 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa0e919b00 con 0x55aa0e855000 2024-09-13T23:43:28.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51be57a700 1 -- 172.21.15.167:0/2237075342 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0e8f9600 con 0x55aa0e855000 2024-09-13T23:43:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2237075342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 msgr2=0x55aa0eb3a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2237075342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 0x55aa0eb3a580 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x55aa0e9b1e90 tx=0x55aa0e959b60 comp rx=0 tx=0).stop 2024-09-13T23:43:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2237075342 shutdown_connections 2024-09-13T23:43:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2237075342 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 0x55aa0eb3a580 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2237075342 >> 172.21.15.167:0/2237075342 conn(0x55aa0e854c00 msgr2=0x55aa0e955000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2237075342 shutdown_connections 2024-09-13T23:43:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2237075342 wait complete. 2024-09-13T23:43:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51c15c4f40 1 Processor -- start 2024-09-13T23:43:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51c15c4f40 1 -- start start 2024-09-13T23:43:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51c15c4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 0x55aa0eb3a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51c15c4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0e8e23c0 con 0x55aa0e855000 2024-09-13T23:43:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51bfd7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 0x55aa0eb3a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51bfd7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 0x55aa0eb3a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51356/0 (socket says 172.21.15.167:51356) 2024-09-13T23:43:28.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.787+0000 7f51bfd7d700 1 -- 172.21.15.167:0/70528131 learned_addr learned my addr 172.21.15.167:0/70528131 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51bfd7d700 1 -- 172.21.15.167:0/70528131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa0e867a00 con 0x55aa0e855000 2024-09-13T23:43:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51bfd7d700 1 --2- 172.21.15.167:0/70528131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 0x55aa0eb3a580 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55aa0f6bc0c0 tx=0x55aa0e959a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51bcd77700 1 -- 172.21.15.167:0/70528131 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0e8f98c0 con 0x55aa0e855000 2024-09-13T23:43:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51bcd77700 1 -- 172.21.15.167:0/70528131 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa0e919c80 con 0x55aa0e855000 2024-09-13T23:43:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51c15c4f40 1 -- 172.21.15.167:0/70528131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55aa0e867860 con 0x55aa0e855000 2024-09-13T23:43:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51bcd77700 1 -- 172.21.15.167:0/70528131 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0e8f94a0 con 0x55aa0e855000 2024-09-13T23:43:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51c15c4f40 1 -- 172.21.15.167:0/70528131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa0e8676c0 con 0x55aa0e855000 2024-09-13T23:43:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51c15c4f40 1 Processor -- start 2024-09-13T23:43:28.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51c15c4f40 1 -- start start 2024-09-13T23:43:28.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51c15c4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 0x55aa0eb3ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51c15c4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0e8e2000 con 0x55aa0e855800 2024-09-13T23:43:28.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51c057e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 0x55aa0eb3ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51c057e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 0x55aa0eb3ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51370/0 (socket says 172.21.15.167:51370) 2024-09-13T23:43:28.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51c057e700 1 -- 172.21.15.167:0/2845843885 learned_addr learned my addr 172.21.15.167:0/2845843885 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51bcd77700 1 -- 172.21.15.167:0/70528131 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55aa0e94eb00 con 0x55aa0e855000 2024-09-13T23:43:28.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51c057e700 1 -- 172.21.15.167:0/2845843885 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa0e866000 con 0x55aa0e855800 2024-09-13T23:43:28.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51c057e700 1 --2- 172.21.15.167:0/2845843885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 0x55aa0eb3ab00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55aa0f6bd350 tx=0x55aa0ebc2660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae75fc64ce456457 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51bcd77700 1 --2- 172.21.15.167:0/70528131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ebae000 0x55aa0eb3b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51bcd77700 1 -- 172.21.15.167:0/70528131 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55aa0e8bb180 con 0x55aa0e855000 2024-09-13T23:43:28.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51ba572700 1 -- 172.21.15.167:0/2845843885 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0e8f98c0 con 0x55aa0e855800 2024-09-13T23:43:28.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51bfd7d700 1 --2- 172.21.15.167:0/70528131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ebae000 0x55aa0eb3b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51ba572700 1 -- 172.21.15.167:0/2845843885 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa0ec0a000 con 0x55aa0e855800 2024-09-13T23:43:28.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.791+0000 7f51ba572700 1 -- 172.21.15.167:0/2845843885 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0e8f94a0 con 0x55aa0e855800 2024-09-13T23:43:28.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2845843885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 msgr2=0x55aa0eb3ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51bfd7d700 1 --2- 172.21.15.167:0/70528131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ebae000 0x55aa0eb3b080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55aa0f6bd710 tx=0x55aa0ebc2ea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2845843885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 0x55aa0eb3ab00 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55aa0f6bd350 tx=0x55aa0ebc2660 comp rx=0 tx=0).stop 2024-09-13T23:43:28.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2845843885 shutdown_connections 2024-09-13T23:43:28.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2845843885 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 0x55aa0eb3ab00 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2845843885 >> 172.21.15.167:0/2845843885 conn(0x55aa0e855400 msgr2=0x55aa0e955800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2845843885 shutdown_connections 2024-09-13T23:43:28.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2845843885 wait complete. 2024-09-13T23:43:28.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 Processor -- start 2024-09-13T23:43:28.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 -- start start 2024-09-13T23:43:28.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 0x55aa0eb3ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0e87a640 con 0x55aa0e855800 2024-09-13T23:43:28.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c057e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 0x55aa0eb3ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c057e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 0x55aa0eb3ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51380/0 (socket says 172.21.15.167:51380) 2024-09-13T23:43:28.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 learned_addr learned my addr 172.21.15.167:0/2857294741 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa0e866ea0 con 0x55aa0e855800 2024-09-13T23:43:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c057e700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 0x55aa0eb3ab00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55aa0f6bced0 tx=0x55aa0e959da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51b8d6f700 1 -- 172.21.15.167:0/2857294741 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0e8f9b80 con 0x55aa0e855800 2024-09-13T23:43:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51b8d6f700 1 -- 172.21.15.167:0/2857294741 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa0ec0a180 con 0x55aa0e855800 2024-09-13T23:43:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55aa0e866820 con 0x55aa0e855800 2024-09-13T23:43:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa0e8669c0 con 0x55aa0e855800 2024-09-13T23:43:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.795+0000 7f51b8d6f700 1 -- 172.21.15.167:0/2857294741 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0e8f98c0 con 0x55aa0e855800 2024-09-13T23:43:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51c15c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51b8d6f700 1 -- 172.21.15.167:0/2857294741 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55aa0e94edc0 con 0x55aa0e855800 2024-09-13T23:43:28.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51b8d6f700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ebaec00 0x55aa0eb3b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51b8d6f700 1 -- 172.21.15.167:0/2857294741 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55aa0e8bb340 con 0x55aa0e855800 2024-09-13T23:43:28.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55aa0ebaf000 0x55aa0eb3bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51bf57c700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ebaec00 0x55aa0eb3b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf400 con 0x55aa0ebaf000 2024-09-13T23:43:28.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51c057e700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55aa0ebaf000 0x55aa0eb3bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51bf57c700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ebaec00 0x55aa0eb3b600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55aa0eb35920 tx=0x55aa0e959800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51c057e700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55aa0ebaf000 0x55aa0eb3bb80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51c15c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51c15c4f40 20 realm 2024-09-13T23:43:28.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51c15c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.799+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf000 2024-09-13T23:43:28.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebafc00 con 0x55aa0ebaf000 2024-09-13T23:43:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55aa0ebaf400 0x55aa0eb3c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf400 2024-09-13T23:43:28.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51bfd7d700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55aa0ebaf400 0x55aa0eb3c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51bfd7d700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55aa0ebaf400 0x55aa0eb3c100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:28.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebafc00 con 0x55aa0ebaf400 2024-09-13T23:43:28.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 20 searching for the correct realm 2024-09-13T23:43:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55aa0eb74000 0x55aa0eb3c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.803+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb74400 con 0x55aa0eb74000 2024-09-13T23:43:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.807+0000 7f51bf57c700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55aa0eb74000 0x55aa0eb3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.807+0000 7f51bf57c700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55aa0eb74000 0x55aa0eb3c680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.807+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.807+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb74400 con 0x55aa0ebaf400 2024-09-13T23:43:28.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.807+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.807+0000 7f51bfd7d700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55aa0eb74400 0x55aa0eb3cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.807+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74400 2024-09-13T23:43:28.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.807+0000 7f51c057e700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55aa0eb74400 0x55aa0eb3cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.807+0000 7f51c057e700 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55aa0eb74400 0x55aa0eb3cc00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.807+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.807+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf000 2024-09-13T23:43:28.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf000 2024-09-13T23:43:28.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74000 2024-09-13T23:43:28.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf000 2024-09-13T23:43:28.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf400 2024-09-13T23:43:28.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf400 2024-09-13T23:43:28.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf000 2024-09-13T23:43:28.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.811+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74400 2024-09-13T23:43:28.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf000 2024-09-13T23:43:28.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74400 2024-09-13T23:43:28.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf400 2024-09-13T23:43:28.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74000 2024-09-13T23:43:28.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf000 2024-09-13T23:43:28.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf400 2024-09-13T23:43:28.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.815+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74000 2024-09-13T23:43:28.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.819+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.819+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf000 2024-09-13T23:43:28.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.819+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.819+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf000 2024-09-13T23:43:28.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.819+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.819+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf000 2024-09-13T23:43:28.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.819+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.819+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74000 2024-09-13T23:43:28.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.819+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.819+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74400 2024-09-13T23:43:28.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.819+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.819+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf000 2024-09-13T23:43:28.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.823+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.823+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74400 2024-09-13T23:43:28.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.823+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.823+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74000 2024-09-13T23:43:28.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.823+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.823+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74400 2024-09-13T23:43:28.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.823+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.823+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf400 2024-09-13T23:43:28.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.823+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.823+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf400 2024-09-13T23:43:28.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.823+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.823+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74000 2024-09-13T23:43:28.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf400 2024-09-13T23:43:28.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf400 2024-09-13T23:43:28.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51c15c4f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:28.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51c15c4f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:28.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51c15c4f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:28.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51c15c4f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:28.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51c15c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb74800 con 0x55aa0ebaf000 2024-09-13T23:43:28.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51c15c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51c15c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.827+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb74c00 con 0x55aa0ebaf400 2024-09-13T23:43:28.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:28.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb75000 con 0x55aa0ebaf400 2024-09-13T23:43:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 zone default found 2024-09-13T23:43:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 4 Realm: () 2024-09-13T23:43:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 zonegroup default 2024-09-13T23:43:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb75400 con 0x55aa0eb74400 2024-09-13T23:43:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb75800 con 0x55aa0eb74000 2024-09-13T23:43:28.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb75c00 con 0x55aa0eb74000 2024-09-13T23:43:28.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:28.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 started sync module instance, tier type = 2024-09-13T23:43:28.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:28.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.831+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb75800 con 0x55aa0ebaf400 2024-09-13T23:43:28.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.835+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.835+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94188879420160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb75c00 con 0x55aa0ebaf400 2024-09-13T23:43:28.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.835+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb74c00 con 0x55aa0ebaf000 2024-09-13T23:43:28.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.839+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94188879420160] v33'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.839+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.839+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94188879421312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb75000 con 0x55aa0ebaf000 2024-09-13T23:43:28.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.839+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb75800 con 0x55aa0ebaf400 2024-09-13T23:43:28.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.843+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94188879421312] v33'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.843+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.843+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94188879422464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebafc00 con 0x55aa0ebaf400 2024-09-13T23:43:28.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.843+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0eb74400 2024-09-13T23:43:28.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.843+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.843+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94188879423616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb74800 con 0x55aa0eb74400 2024-09-13T23:43:28.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.843+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb75400 con 0x55aa0ebaf400 2024-09-13T23:43:28.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.847+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94188879422464] v33'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.847+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94188879423616] v33'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.847+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.847+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94188879424768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb74c00 con 0x55aa0ebaf400 2024-09-13T23:43:28.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.847+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb75000 con 0x55aa0eb74000 2024-09-13T23:43:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.851+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.851+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94188879424768] v33'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.851+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94188879425920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ecc8000 con 0x55aa0eb74000 2024-09-13T23:43:28.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.851+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ecc8400 con 0x55aa0eb74000 2024-09-13T23:43:28.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.851+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.851+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94188879427072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ecc8800 con 0x55aa0eb74000 2024-09-13T23:43:28.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.851+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ecc8c00 con 0x55aa0eb74400 2024-09-13T23:43:28.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.851+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94188879425920] v33'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94188879427072] v33'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'69 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94188879428224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ecc9000 con 0x55aa0eb74400 2024-09-13T23:43:28.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c15c4f40 20 add_watcher() i=0 2024-09-13T23:43:28.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c15c4f40 20 add_watcher() i=1 2024-09-13T23:43:28.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c15c4f40 20 add_watcher() i=2 2024-09-13T23:43:28.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c15c4f40 20 add_watcher() i=3 2024-09-13T23:43:28.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c15c4f40 20 add_watcher() i=4 2024-09-13T23:43:28.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c15c4f40 20 add_watcher() i=5 2024-09-13T23:43:28.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c15c4f40 20 add_watcher() i=6 2024-09-13T23:43:28.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94188879428224] v33'70 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c15c4f40 20 add_watcher() i=7 2024-09-13T23:43:28.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c15c4f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:28.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ecc9400 con 0x55aa0ebaf000 2024-09-13T23:43:28.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.855+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94188881108992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ecc9800 con 0x55aa0ebaf000 2024-09-13T23:43:28.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.859+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94188881108992] v33'80 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.859+0000 7f51a6d4b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:28.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.859+0000 7f51c15c4f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:28.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.859+0000 7f51c15c4f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:28.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.859+0000 7f51c15c4f40 1 Processor -- start 2024-09-13T23:43:28.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.859+0000 7f51c15c4f40 1 -- start start 2024-09-13T23:43:28.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.859+0000 7f51c15c4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 0x55aa0eb3d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.859+0000 7f51c15c4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0e87a780 con 0x55aa0ecc8c00 2024-09-13T23:43:28.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.859+0000 7f51bf57c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 0x55aa0eb3d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.859+0000 7f51bf57c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 0x55aa0eb3d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51388/0 (socket says 172.21.15.167:51388) 2024-09-13T23:43:28.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.859+0000 7f51bf57c700 1 -- 172.21.15.167:0/1663118621 learned_addr learned my addr 172.21.15.167:0/1663118621 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.859+0000 7f51bf57c700 1 -- 172.21.15.167:0/1663118621 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa0e866b60 con 0x55aa0ecc8c00 2024-09-13T23:43:28.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51bf57c700 1 --2- 172.21.15.167:0/1663118621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 0x55aa0eb3d180 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55aa0ec5d410 tx=0x55aa0e959860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb3b88a59638bc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51a654a700 1 -- 172.21.15.167:0/1663118621 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0e8f9760 con 0x55aa0ecc8c00 2024-09-13T23:43:28.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51a654a700 1 -- 172.21.15.167:0/1663118621 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa0ec0a600 con 0x55aa0ecc8c00 2024-09-13T23:43:28.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51a654a700 1 -- 172.21.15.167:0/1663118621 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0ecdc000 con 0x55aa0ecc8c00 2024-09-13T23:43:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51c15c4f40 1 -- 172.21.15.167:0/1663118621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 msgr2=0x55aa0eb3d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/1663118621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 0x55aa0eb3d180 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55aa0ec5d410 tx=0x55aa0e959860 comp rx=0 tx=0).stop 2024-09-13T23:43:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51c15c4f40 1 -- 172.21.15.167:0/1663118621 shutdown_connections 2024-09-13T23:43:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/1663118621 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 0x55aa0eb3d180 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51c15c4f40 1 -- 172.21.15.167:0/1663118621 >> 172.21.15.167:0/1663118621 conn(0x55aa0ecc9c00 msgr2=0x55aa0e956000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51c15c4f40 1 -- 172.21.15.167:0/1663118621 shutdown_connections 2024-09-13T23:43:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51c15c4f40 1 -- 172.21.15.167:0/1663118621 wait complete. 2024-09-13T23:43:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51c15c4f40 1 Processor -- start 2024-09-13T23:43:28.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51c15c4f40 1 -- start start 2024-09-13T23:43:28.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51c15c4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 0x55aa0eb3d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51c15c4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0e8e2000 con 0x55aa0ecc8c00 2024-09-13T23:43:28.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51bf57c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 0x55aa0eb3d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51bf57c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 0x55aa0eb3d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51390/0 (socket says 172.21.15.167:51390) 2024-09-13T23:43:28.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51bf57c700 1 -- 172.21.15.167:0/3736976230 learned_addr learned my addr 172.21.15.167:0/3736976230 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:28.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.863+0000 7f51bf57c700 1 -- 172.21.15.167:0/3736976230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa0e866d00 con 0x55aa0ecc8c00 2024-09-13T23:43:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.867+0000 7f51bf57c700 1 --2- 172.21.15.167:0/3736976230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 0x55aa0eb3d180 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55aa0ecd81b0 tx=0x55aa0ecda300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.867+0000 7f51a4d47700 1 -- 172.21.15.167:0/3736976230 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0ecdc160 con 0x55aa0ecc8c00 2024-09-13T23:43:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.867+0000 7f51a4d47700 1 -- 172.21.15.167:0/3736976230 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aa0ec0a300 con 0x55aa0ecc8c00 2024-09-13T23:43:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.867+0000 7f51c15c4f40 1 -- 172.21.15.167:0/3736976230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55aa0e866340 con 0x55aa0ecc8c00 2024-09-13T23:43:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.867+0000 7f51a4d47700 1 -- 172.21.15.167:0/3736976230 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55aa0ecdc2c0 con 0x55aa0ecc8c00 2024-09-13T23:43:28.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.867+0000 7f51c15c4f40 1 -- 172.21.15.167:0/3736976230 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa0e867d40 con 0x55aa0ecc8c00 2024-09-13T23:43:28.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.867+0000 7f51a4d47700 1 -- 172.21.15.167:0/3736976230 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55aa0e94fb80 con 0x55aa0ecc8c00 2024-09-13T23:43:28.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.867+0000 7f51a4d47700 1 --2- 172.21.15.167:0/3736976230 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ed54000 0x55aa0ed5a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:28.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.867+0000 7f51a4d47700 1 -- 172.21.15.167:0/3736976230 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55aa0e8bb500 con 0x55aa0ecc8c00 2024-09-13T23:43:28.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.867+0000 7f51c057e700 1 --2- 172.21.15.167:0/3736976230 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ed54000 0x55aa0ed5a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:28.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.867+0000 7f51c15c4f40 5 note: GC not initialized 2024-09-13T23:43:28.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.867+0000 7f51c057e700 1 --2- 172.21.15.167:0/3736976230 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ed54000 0x55aa0ed5a000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55aa0ec5dc80 tx=0x55aa0ebc3680 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.871+0000 7f51a2542700 20 reqs_thread_entry: start 2024-09-13T23:43:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.871+0000 7f51c15c4f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:28.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.871+0000 7f51c15c4f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:28.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.871+0000 7f51a053e700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:59 2024 2024-09-13T23:43:28.880 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:28.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.871+0000 7f519fd1c700 20 reqs_thread_entry: start 2024-09-13T23:43:28.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.871+0000 7f51c15c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:28.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.871+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ecc8000 con 0x55aa0ebaf000 2024-09-13T23:43:28.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.871+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~621 out=621b,getxattrs out=75b] v0'0 uv8 ondisk = 0) v8 ==== 231+0+744 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.871+0000 7f51c15c4f40 20 rados_obj.operate() r=0 bl.length=621 2024-09-13T23:43:28.881 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:28.881 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:28.881 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:28.881 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:28.881 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:28.881 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:28.881 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [ 2024-09-13T23:43:28.881 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.881 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "foo:foo1", 2024-09-13T23:43:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: "permissions": "full-control" 2024-09-13T23:43:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "foo:foo2", 2024-09-13T23:43:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: "permissions": "" 2024-09-13T23:43:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [ 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo:foo1", 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL" 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo:foo2", 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:28.885 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:28.885 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:28.885 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:28.885 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:28.885 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:28.885 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:28.885 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:28.885 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:28.885 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:28.885 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:28.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.871+0000 7f51a6d4b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:28.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.871+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94188881108992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ecc8400 con 0x55aa0ebaf000 2024-09-13T23:43:28.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.875+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94188881108992] v33'81 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.875+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94188879420160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb74800 con 0x55aa0ebaf400 2024-09-13T23:43:28.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.879+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94188879420160] v33'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.879+0000 7f51c15c4f40 20 remove_watcher() i=0 2024-09-13T23:43:28.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.879+0000 7f51c15c4f40 2 removed watcher, disabling cache 2024-09-13T23:43:28.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.879+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94188879421312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0eb75000 con 0x55aa0ebaf000 2024-09-13T23:43:28.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.879+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94188879421312] v33'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0ebaf000 2024-09-13T23:43:28.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.879+0000 7f51c15c4f40 20 remove_watcher() i=1 2024-09-13T23:43:28.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.879+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94188879422464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ebaf800 con 0x55aa0ebaf400 2024-09-13T23:43:28.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.883+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94188879422464] v33'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.883+0000 7f51c15c4f40 20 remove_watcher() i=2 2024-09-13T23:43:28.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.883+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94188879423616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ed54400 con 0x55aa0eb74400 2024-09-13T23:43:28.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.887+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94188879423616] v33'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.887+0000 7f51c15c4f40 20 remove_watcher() i=3 2024-09-13T23:43:28.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.887+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94188879424768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ed54800 con 0x55aa0ebaf400 2024-09-13T23:43:28.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.887+0000 7f51bfd7d700 1 -- 172.21.15.167:0/2857294741 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94188879424768] v33'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65200 con 0x55aa0ebaf400 2024-09-13T23:43:28.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.887+0000 7f51c15c4f40 20 remove_watcher() i=4 2024-09-13T23:43:28.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.887+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94188879425920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ed54c00 con 0x55aa0eb74000 2024-09-13T23:43:28.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.891+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94188879425920] v33'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.891+0000 7f51c15c4f40 20 remove_watcher() i=5 2024-09-13T23:43:28.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.891+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94188879427072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ed55000 con 0x55aa0eb74000 2024-09-13T23:43:28.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.891+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94188879427072] v33'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec65440 con 0x55aa0eb74000 2024-09-13T23:43:28.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.891+0000 7f51c15c4f40 20 remove_watcher() i=6 2024-09-13T23:43:28.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.891+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94188879428224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55aa0ed55400 con 0x55aa0eb74400 2024-09-13T23:43:28.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c057e700 1 -- 172.21.15.167:0/2857294741 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94188879428224] v33'71 uv69 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55aa0ec64b40 con 0x55aa0eb74400 2024-09-13T23:43:28.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 20 remove_watcher() i=7 2024-09-13T23:43:28.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55aa0ebaf400 msgr2=0x55aa0eb3c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55aa0ebaf400 0x55aa0eb3c100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55aa0ebaf000 msgr2=0x55aa0eb3bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55aa0ebaf000 0x55aa0eb3bb80 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55aa0eb74400 msgr2=0x55aa0eb3cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55aa0eb74400 0x55aa0eb3cc00 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55aa0eb74000 msgr2=0x55aa0eb3c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55aa0eb74000 0x55aa0eb3c680 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ebaec00 msgr2=0x55aa0eb3b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ebaec00 0x55aa0eb3b600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x55aa0eb35920 tx=0x55aa0e959800 comp rx=0 tx=0).stop 2024-09-13T23:43:28.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 msgr2=0x55aa0eb3ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/2857294741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855800 0x55aa0eb3ab00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x55aa0f6bced0 tx=0x55aa0e959da0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 reap_dead start 2024-09-13T23:43:28.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51bf57c700 1 -- 172.21.15.167:0/2857294741 reap_dead start 2024-09-13T23:43:28.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 shutdown_connections 2024-09-13T23:43:28.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 >> 172.21.15.167:0/2857294741 conn(0x55aa0e855400 msgr2=0x55aa0e955800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 shutdown_connections 2024-09-13T23:43:28.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/2857294741 wait complete. 2024-09-13T23:43:28.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/3736976230 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ed54000 msgr2=0x55aa0ed5a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/3736976230 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ed54000 0x55aa0ed5a000 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x55aa0ec5dc80 tx=0x55aa0ebc3680 comp rx=0 tx=0).stop 2024-09-13T23:43:28.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/3736976230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 msgr2=0x55aa0eb3d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/3736976230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 0x55aa0eb3d180 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55aa0ecd81b0 tx=0x55aa0ecda300 comp rx=0 tx=0).stop 2024-09-13T23:43:28.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/3736976230 shutdown_connections 2024-09-13T23:43:28.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/3736976230 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ed54000 0x55aa0ed5a000 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/3736976230 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0ecc8c00 0x55aa0eb3d180 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/3736976230 >> 172.21.15.167:0/3736976230 conn(0x55aa0ecc9c00 msgr2=0x55aa0e956000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/3736976230 shutdown_connections 2024-09-13T23:43:28.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.895+0000 7f51c15c4f40 1 -- 172.21.15.167:0/3736976230 wait complete. 2024-09-13T23:43:28.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.899+0000 7f51c15c4f40 1 -- 172.21.15.167:0/70528131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ebae000 msgr2=0x55aa0eb3b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.899+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/70528131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ebae000 0x55aa0eb3b080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x55aa0f6bd710 tx=0x55aa0ebc2ea0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.899+0000 7f51c15c4f40 1 -- 172.21.15.167:0/70528131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 msgr2=0x55aa0eb3a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:28.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.899+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/70528131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 0x55aa0eb3a580 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x55aa0f6bc0c0 tx=0x55aa0e959a40 comp rx=0 tx=0).stop 2024-09-13T23:43:28.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.899+0000 7f51c15c4f40 1 -- 172.21.15.167:0/70528131 shutdown_connections 2024-09-13T23:43:28.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.899+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/70528131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55aa0ebae000 0x55aa0eb3b080 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.899+0000 7f51c15c4f40 1 --2- 172.21.15.167:0/70528131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55aa0e855000 0x55aa0eb3a580 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:28.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.899+0000 7f51c15c4f40 1 -- 172.21.15.167:0/70528131 >> 172.21.15.167:0/70528131 conn(0x55aa0e854c00 msgr2=0x55aa0e955000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:28.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.899+0000 7f51c15c4f40 1 -- 172.21.15.167:0/70528131 shutdown_connections 2024-09-13T23:43:28.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.899+0000 7f51c15c4f40 1 -- 172.21.15.167:0/70528131 wait complete. 2024-09-13T23:43:28.912 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:28.913 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-09-13T23:43:28.913 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-09-13T23:43:28.913 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --subuser foo:foo1 --key-type swift 2024-09-13T23:43:28.999 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:28.999 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:29.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f4342f40 1 Processor -- start 2024-09-13T23:43:29.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f4342f40 1 -- start start 2024-09-13T23:43:29.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f4342f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56173b22f400 0x56173b514580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f4342f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56173b2f3500 con 0x56173b22f000 2024-09-13T23:43:29.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f4342f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x56173b2bca00 con 0x56173b22f400 2024-09-13T23:43:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f22fa700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56173b22f400 0x56173b514580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f2afb700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x56173b22f000 0x56173b32e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45346/0 (socket says 172.21.15.167:45346) 2024-09-13T23:43:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f22fa700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56173b22f400 0x56173b514580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51402/0 (socket says 172.21.15.167:51402) 2024-09-13T23:43:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f2afb700 1 -- 172.21.15.167:0/3855670321 learned_addr learned my addr 172.21.15.167:0/3855670321 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:29.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f22fa700 1 -- 172.21.15.167:0/3855670321 >> v1:172.21.15.12:6789/0 conn(0x56173b22f000 legacy=0x56173b32e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f22fa700 1 -- 172.21.15.167:0/3855670321 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173b241d40 con 0x56173b22f400 2024-09-13T23:43:29.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f22fa700 1 --2- 172.21.15.167:0/3855670321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f400 0x56173b514580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x56173b50f920 tx=0x56173b333920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20efe0b94556458f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f1af9700 1 -- 172.21.15.167:0/3855670321 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b2d34a0 con 0x56173b22f400 2024-09-13T23:43:29.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f1af9700 1 -- 172.21.15.167:0/3855670321 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56173b2f3800 con 0x56173b22f400 2024-09-13T23:43:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.991+0000 7ff9f1af9700 1 -- 172.21.15.167:0/3855670321 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b2d3600 con 0x56173b22f400 2024-09-13T23:43:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3855670321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f400 msgr2=0x56173b514580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/3855670321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f400 0x56173b514580 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x56173b50f920 tx=0x56173b333920 comp rx=0 tx=0).stop 2024-09-13T23:43:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3855670321 shutdown_connections 2024-09-13T23:43:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/3855670321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f400 0x56173b514580 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3855670321 >> 172.21.15.167:0/3855670321 conn(0x56173b22ec00 msgr2=0x56173b32e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3855670321 shutdown_connections 2024-09-13T23:43:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3855670321 wait complete. 2024-09-13T23:43:29.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 Processor -- start 2024-09-13T23:43:29.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 -- start start 2024-09-13T23:43:29.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 0x56173b514580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173b2bc500 con 0x56173b22f000 2024-09-13T23:43:29.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f2afb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 0x56173b514580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f2afb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 0x56173b514580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51414/0 (socket says 172.21.15.167:51414) 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f2afb700 1 -- 172.21.15.167:0/62542917 learned_addr learned my addr 172.21.15.167:0/62542917 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f2afb700 1 -- 172.21.15.167:0/62542917 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173b241ba0 con 0x56173b22f000 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f2afb700 1 --2- 172.21.15.167:0/62542917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 0x56173b514580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x56173b38be00 tx=0x56173b333b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf70b9e525b95077 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f12f8700 1 -- 172.21.15.167:0/62542917 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b2d34a0 con 0x56173b22f000 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f12f8700 1 -- 172.21.15.167:0/62542917 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56173b2f3b00 con 0x56173b22f000 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f12f8700 1 -- 172.21.15.167:0/62542917 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b2d3600 con 0x56173b22f000 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 -- 172.21.15.167:0/62542917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 msgr2=0x56173b514580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/62542917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 0x56173b514580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x56173b38be00 tx=0x56173b333b60 comp rx=0 tx=0).stop 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 -- 172.21.15.167:0/62542917 shutdown_connections 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/62542917 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 0x56173b514580 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 -- 172.21.15.167:0/62542917 >> 172.21.15.167:0/62542917 conn(0x56173b22ec00 msgr2=0x56173b32f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 -- 172.21.15.167:0/62542917 shutdown_connections 2024-09-13T23:43:29.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.995+0000 7ff9f4342f40 1 -- 172.21.15.167:0/62542917 wait complete. 2024-09-13T23:43:29.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9f4342f40 1 Processor -- start 2024-09-13T23:43:29.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9f4342f40 1 -- start start 2024-09-13T23:43:29.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9f4342f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 0x56173b514580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9f4342f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173b2bc3c0 con 0x56173b22f000 2024-09-13T23:43:29.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9f2afb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 0x56173b514580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9f2afb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 0x56173b514580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51420/0 (socket says 172.21.15.167:51420) 2024-09-13T23:43:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9f2afb700 1 -- 172.21.15.167:0/2947051324 learned_addr learned my addr 172.21.15.167:0/2947051324 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9f2afb700 1 -- 172.21.15.167:0/2947051324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173b241a00 con 0x56173b22f000 2024-09-13T23:43:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9f2afb700 1 --2- 172.21.15.167:0/2947051324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 0x56173b514580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x56173b38bf20 tx=0x56173b333a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9efaf5700 1 -- 172.21.15.167:0/2947051324 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b2d38c0 con 0x56173b22f000 2024-09-13T23:43:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9f4342f40 1 -- 172.21.15.167:0/2947051324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56173b241860 con 0x56173b22f000 2024-09-13T23:43:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9efaf5700 1 -- 172.21.15.167:0/2947051324 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56173b2f3c80 con 0x56173b22f000 2024-09-13T23:43:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9efaf5700 1 -- 172.21.15.167:0/2947051324 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b2d34a0 con 0x56173b22f000 2024-09-13T23:43:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9f4342f40 1 -- 172.21.15.167:0/2947051324 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56173b2416c0 con 0x56173b22f000 2024-09-13T23:43:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:28.999+0000 7ff9f4342f40 1 Processor -- start 2024-09-13T23:43:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 -- start start 2024-09-13T23:43:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9efaf5700 1 -- 172.21.15.167:0/2947051324 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56173b328b00 con 0x56173b22f000 2024-09-13T23:43:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 0x56173b514b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173b254640 con 0x56173b22f800 2024-09-13T23:43:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9efaf5700 1 --2- 172.21.15.167:0/2947051324 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b5d8000 0x56173b515080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9efaf5700 1 -- 172.21.15.167:0/2947051324 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56173b295180 con 0x56173b22f000 2024-09-13T23:43:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f2afb700 1 --2- 172.21.15.167:0/2947051324 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b5d8000 0x56173b515080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f32fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 0x56173b514b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f32fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 0x56173b514b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51422/0 (socket says 172.21.15.167:51422) 2024-09-13T23:43:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f32fc700 1 -- 172.21.15.167:0/325293698 learned_addr learned my addr 172.21.15.167:0/325293698 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f32fc700 1 -- 172.21.15.167:0/325293698 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173b240000 con 0x56173b22f800 2024-09-13T23:43:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f2afb700 1 --2- 172.21.15.167:0/2947051324 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b5d8000 0x56173b515080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x56173b55af60 tx=0x56173b592240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f32fc700 1 --2- 172.21.15.167:0/325293698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 0x56173b514b00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x56173b55a240 tx=0x56173b5930e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdf08616fc51e95 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9ed2f0700 1 -- 172.21.15.167:0/325293698 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b2d38c0 con 0x56173b22f800 2024-09-13T23:43:29.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9ed2f0700 1 -- 172.21.15.167:0/325293698 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56173b5ec000 con 0x56173b22f800 2024-09-13T23:43:29.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9ed2f0700 1 -- 172.21.15.167:0/325293698 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b2d34a0 con 0x56173b22f800 2024-09-13T23:43:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 -- 172.21.15.167:0/325293698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 msgr2=0x56173b514b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/325293698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 0x56173b514b00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x56173b55a240 tx=0x56173b5930e0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 -- 172.21.15.167:0/325293698 shutdown_connections 2024-09-13T23:43:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/325293698 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 0x56173b514b00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 -- 172.21.15.167:0/325293698 >> 172.21.15.167:0/325293698 conn(0x56173b22f400 msgr2=0x56173b32f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 -- 172.21.15.167:0/325293698 shutdown_connections 2024-09-13T23:43:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 -- 172.21.15.167:0/325293698 wait complete. 2024-09-13T23:43:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 Processor -- start 2024-09-13T23:43:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 -- start start 2024-09-13T23:43:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 0x56173b514b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.003+0000 7ff9f4342f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173b2548c0 con 0x56173b22f800 2024-09-13T23:43:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9f32fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 0x56173b514b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9f32fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 0x56173b514b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51424/0 (socket says 172.21.15.167:51424) 2024-09-13T23:43:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 learned_addr learned my addr 172.21.15.167:0/499819871 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173b240ea0 con 0x56173b22f800 2024-09-13T23:43:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9f32fc700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 0x56173b514b00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56173b38bfb0 tx=0x56173b592cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9ebaed700 1 -- 172.21.15.167:0/499819871 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b2d3b80 con 0x56173b22f800 2024-09-13T23:43:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9ebaed700 1 -- 172.21.15.167:0/499819871 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56173b5ec180 con 0x56173b22f800 2024-09-13T23:43:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56173b240820 con 0x56173b22f800 2024-09-13T23:43:29.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9ebaed700 1 -- 172.21.15.167:0/499819871 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b2d38c0 con 0x56173b22f800 2024-09-13T23:43:29.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56173b2409c0 con 0x56173b22f800 2024-09-13T23:43:29.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9ebaed700 1 -- 172.21.15.167:0/499819871 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56173b329080 con 0x56173b22f800 2024-09-13T23:43:29.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9ebaed700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b5d8c00 0x56173b515600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9ebaed700 1 -- 172.21.15.167:0/499819871 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56173b295340 con 0x56173b22f800 2024-09-13T23:43:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9f4342f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9f22fa700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b5d8c00 0x56173b515600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56173b5d9000 0x56173b515b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.007+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9400 con 0x56173b5d9000 2024-09-13T23:43:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f32fc700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56173b5d9000 0x56173b515b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f22fa700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b5d8c00 0x56173b515600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x56173b50f920 tx=0x56173b333800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f32fc700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56173b5d9000 0x56173b515b80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f4342f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f4342f40 20 realm 2024-09-13T23:43:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f4342f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9000 2024-09-13T23:43:29.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f4342f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f4342f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:29.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f4342f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9c00 con 0x56173b5d9000 2024-09-13T23:43:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f4342f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f4342f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56173b5d9400 0x56173b516100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9400 2024-09-13T23:43:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f2afb700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56173b5d9400 0x56173b516100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.011+0000 7ff9f2afb700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56173b5d9400 0x56173b516100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f4342f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:29.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f4342f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9c00 con 0x56173b5d9400 2024-09-13T23:43:29.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f4342f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:29.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f4342f40 20 searching for the correct realm 2024-09-13T23:43:29.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56173b692000 0x56173b516680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b692400 con 0x56173b692000 2024-09-13T23:43:29.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f22fa700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56173b692000 0x56173b516680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f22fa700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56173b692000 0x56173b516680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b692400 con 0x56173b5d9400 2024-09-13T23:43:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f2afb700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56173b692400 0x56173b516c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692400 2024-09-13T23:43:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.015+0000 7ff9f32fc700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56173b692400 0x56173b516c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f32fc700 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56173b692400 0x56173b516c00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9000 2024-09-13T23:43:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9000 2024-09-13T23:43:29.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692000 2024-09-13T23:43:29.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9000 2024-09-13T23:43:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9400 2024-09-13T23:43:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9400 2024-09-13T23:43:29.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.019+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9000 2024-09-13T23:43:29.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692400 2024-09-13T23:43:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9000 2024-09-13T23:43:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692400 2024-09-13T23:43:29.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9400 2024-09-13T23:43:29.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692000 2024-09-13T23:43:29.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9000 2024-09-13T23:43:29.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.023+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9400 2024-09-13T23:43:29.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692000 2024-09-13T23:43:29.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9000 2024-09-13T23:43:29.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9000 2024-09-13T23:43:29.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9000 2024-09-13T23:43:29.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692000 2024-09-13T23:43:29.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692400 2024-09-13T23:43:29.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9000 2024-09-13T23:43:29.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.027+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692400 2024-09-13T23:43:29.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.031+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.031+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692000 2024-09-13T23:43:29.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.031+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.031+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692400 2024-09-13T23:43:29.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.031+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.031+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9400 2024-09-13T23:43:29.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.031+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.031+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9400 2024-09-13T23:43:29.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.031+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.031+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692000 2024-09-13T23:43:29.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.031+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.031+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9400 2024-09-13T23:43:29.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9400 2024-09-13T23:43:29.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:29.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:29.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:29.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:29.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b692800 con 0x56173b5d9000 2024-09-13T23:43:29.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b692c00 con 0x56173b5d9400 2024-09-13T23:43:29.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:29.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693000 con 0x56173b5d9400 2024-09-13T23:43:29.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:29.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 zone default found 2024-09-13T23:43:29.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 4 Realm: () 2024-09-13T23:43:29.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:29.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:29.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:29.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 zonegroup default 2024-09-13T23:43:29.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693400 con 0x56173b692400 2024-09-13T23:43:29.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.035+0000 7ff9f4342f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.039+0000 7ff9f4342f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.039+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693800 con 0x56173b692000 2024-09-13T23:43:29.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.039+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.039+0000 7ff9f4342f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.039+0000 7ff9f4342f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.039+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693c00 con 0x56173b692000 2024-09-13T23:43:29.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.039+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.039+0000 7ff9f4342f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.039+0000 7ff9f4342f40 20 started sync module instance, tier type = 2024-09-13T23:43:29.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.039+0000 7ff9f4342f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:29.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.039+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693800 con 0x56173b5d9400 2024-09-13T23:43:29.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.043+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.043+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94657779383040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693c00 con 0x56173b5d9400 2024-09-13T23:43:29.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.043+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b692c00 con 0x56173b5d9000 2024-09-13T23:43:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.043+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.043+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94657779384192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693000 con 0x56173b5d9000 2024-09-13T23:43:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.043+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693800 con 0x56173b5d9400 2024-09-13T23:43:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.043+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94657779383040] v33'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.047+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94657779384192] v33'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.047+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.047+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94657779385344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9c00 con 0x56173b5d9400 2024-09-13T23:43:29.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.047+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b692400 2024-09-13T23:43:29.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.047+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.047+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94657779386496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b692800 con 0x56173b692400 2024-09-13T23:43:29.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.051+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693400 con 0x56173b5d9400 2024-09-13T23:43:29.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.051+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94657779385344] v33'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.051+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94657779386496] v33'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.051+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.051+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94657779387648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b692c00 con 0x56173b5d9400 2024-09-13T23:43:29.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.051+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693000 con 0x56173b692000 2024-09-13T23:43:29.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.055+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.055+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94657779388800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b6a0000 con 0x56173b692000 2024-09-13T23:43:29.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.055+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b6a0400 con 0x56173b692000 2024-09-13T23:43:29.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.055+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94657779387648] v33'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.055+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94657779388800] v33'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.055+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.055+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94657779389952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b6a0800 con 0x56173b692000 2024-09-13T23:43:29.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.055+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b6a0c00 con 0x56173b692400 2024-09-13T23:43:29.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94657779389952] v33'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'72 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94657779391104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b6a1000 con 0x56173b692400 2024-09-13T23:43:29.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f4342f40 20 add_watcher() i=0 2024-09-13T23:43:29.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f4342f40 20 add_watcher() i=1 2024-09-13T23:43:29.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f4342f40 20 add_watcher() i=2 2024-09-13T23:43:29.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f4342f40 20 add_watcher() i=3 2024-09-13T23:43:29.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f4342f40 20 add_watcher() i=4 2024-09-13T23:43:29.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f4342f40 20 add_watcher() i=5 2024-09-13T23:43:29.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f4342f40 20 add_watcher() i=6 2024-09-13T23:43:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94657779391104] v33'73 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f4342f40 20 add_watcher() i=7 2024-09-13T23:43:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f4342f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b6a1400 con 0x56173b5d9000 2024-09-13T23:43:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.059+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94657781063680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b6a1800 con 0x56173b5d9000 2024-09-13T23:43:29.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94657781063680] v33'82 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9d9ac9700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:29.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9f4342f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:29.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9f4342f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:29.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9f4342f40 1 Processor -- start 2024-09-13T23:43:29.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9f4342f40 1 -- start start 2024-09-13T23:43:29.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9f4342f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 0x56173b517180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9f4342f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173b2bc3c0 con 0x56173b6a0c00 2024-09-13T23:43:29.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9f22fa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 0x56173b517180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9f22fa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 0x56173b517180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51432/0 (socket says 172.21.15.167:51432) 2024-09-13T23:43:29.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9f22fa700 1 -- 172.21.15.167:0/3888754277 learned_addr learned my addr 172.21.15.167:0/3888754277 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9f22fa700 1 -- 172.21.15.167:0/3888754277 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173b240b60 con 0x56173b6a0c00 2024-09-13T23:43:29.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.063+0000 7ff9f22fa700 1 --2- 172.21.15.167:0/3888754277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 0x56173b517180 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x56173b6ae0f0 tx=0x56173b333860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45831fc23162334 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9d92c8700 1 -- 172.21.15.167:0/3888754277 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b2d3760 con 0x56173b6a0c00 2024-09-13T23:43:29.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9d92c8700 1 -- 172.21.15.167:0/3888754277 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56173b5ec600 con 0x56173b6a0c00 2024-09-13T23:43:29.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9d92c8700 1 -- 172.21.15.167:0/3888754277 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b6b4000 con 0x56173b6a0c00 2024-09-13T23:43:29.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3888754277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 msgr2=0x56173b517180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/3888754277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 0x56173b517180 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x56173b6ae0f0 tx=0x56173b333860 comp rx=0 tx=0).stop 2024-09-13T23:43:29.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3888754277 shutdown_connections 2024-09-13T23:43:29.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/3888754277 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 0x56173b517180 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3888754277 >> 172.21.15.167:0/3888754277 conn(0x56173b6a1c00 msgr2=0x56173b330000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3888754277 shutdown_connections 2024-09-13T23:43:29.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3888754277 wait complete. 2024-09-13T23:43:29.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f4342f40 1 Processor -- start 2024-09-13T23:43:29.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f4342f40 1 -- start start 2024-09-13T23:43:29.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f4342f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 0x56173b517180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f4342f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56173b2bc500 con 0x56173b6a0c00 2024-09-13T23:43:29.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f22fa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 0x56173b517180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f22fa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 0x56173b517180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51436/0 (socket says 172.21.15.167:51436) 2024-09-13T23:43:29.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f22fa700 1 -- 172.21.15.167:0/3981102410 learned_addr learned my addr 172.21.15.167:0/3981102410 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.067+0000 7ff9f22fa700 1 -- 172.21.15.167:0/3981102410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56173b240d00 con 0x56173b6a0c00 2024-09-13T23:43:29.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.071+0000 7ff9f22fa700 1 --2- 172.21.15.167:0/3981102410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 0x56173b517180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x56173b6ae390 tx=0x56173b6b0300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.071+0000 7ff9d7ac5700 1 -- 172.21.15.167:0/3981102410 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b6b4160 con 0x56173b6a0c00 2024-09-13T23:43:29.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.071+0000 7ff9d7ac5700 1 -- 172.21.15.167:0/3981102410 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56173b5ec300 con 0x56173b6a0c00 2024-09-13T23:43:29.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.071+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3981102410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56173b240340 con 0x56173b6a0c00 2024-09-13T23:43:29.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.071+0000 7ff9d7ac5700 1 -- 172.21.15.167:0/3981102410 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56173b6b42c0 con 0x56173b6a0c00 2024-09-13T23:43:29.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.071+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3981102410 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56173b241d40 con 0x56173b6a0c00 2024-09-13T23:43:29.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.071+0000 7ff9d7ac5700 1 -- 172.21.15.167:0/3981102410 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56173b329b80 con 0x56173b6a0c00 2024-09-13T23:43:29.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.071+0000 7ff9d7ac5700 1 --2- 172.21.15.167:0/3981102410 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b738000 0x56173b73e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.071+0000 7ff9d7ac5700 1 -- 172.21.15.167:0/3981102410 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56173b295500 con 0x56173b6a0c00 2024-09-13T23:43:29.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.071+0000 7ff9f32fc700 1 --2- 172.21.15.167:0/3981102410 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b738000 0x56173b73e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.071+0000 7ff9f4342f40 5 note: GC not initialized 2024-09-13T23:43:29.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.071+0000 7ff9f32fc700 1 --2- 172.21.15.167:0/3981102410 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b738000 0x56173b73e000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x56173b6438c0 tx=0x56173b5937a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9d52c0700 20 reqs_thread_entry: start 2024-09-13T23:43:29.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9f4342f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:29.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9f4342f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:29.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9d32bc700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:59 2024 2024-09-13T23:43:29.084 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:29.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9d2a9a700 20 reqs_thread_entry: start 2024-09-13T23:43:29.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9f4342f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:29.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9f4342f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b6a0000 con 0x56173b5d9000 2024-09-13T23:43:29.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~621 out=621b,getxattrs out=75b] v0'0 uv8 ondisk = 0) v8 ==== 231+0+744 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9f4342f40 20 rados_obj.operate() r=0 bl.length=621 2024-09-13T23:43:29.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9f4342f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9f4342f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9f4342f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.075+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~539 in=539b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b692800 con 0x56173b5d9000 2024-09-13T23:43:29.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~539] v33'9 uv9 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9f4342f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:29.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9f4342f40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:29.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9f4342f40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:29.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94657781065984 in=727b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693000 con 0x56173b5d9400 2024-09-13T23:43:29.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94657779387648 notify 141733920780 ret 0) v3 ==== 757+0+0 (crc 0 0 0) 0x56173b590820 con 0x56173b5d9400 2024-09-13T23:43:29.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9ed2f0700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920780 cookie 94657779387648 notifier 4422 bl.length()=715 2024-09-13T23:43:29.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9ed2f0700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:29.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9ed2f0700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:29.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(64 notify.4 [notify cookie 94657781065984 out=8b] v0'0 uv72 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9ed2f0700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693c00 con 0x56173b5d9400 2024-09-13T23:43:29.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94657781065984 notify 141733920780 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56173b5909c0 con 0x56173b5d9400 2024-09-13T23:43:29.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.079+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9800 con 0x56173b5d9000 2024-09-13T23:43:29.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.083+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(66 foo:foo1 [delete] v33'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.083+0000 7ff9f4342f40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-09-13T23:43:29.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.083+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94657781065984 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b738400 con 0x56173b5d9400 2024-09-13T23:43:29.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.083+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== watch-notify(notify (1) cookie 94657779383040 notify 141733920781 ret 0) v3 ==== 201+0+0 (crc 0 0 0) 0x56173b5909c0 con 0x56173b5d9400 2024-09-13T23:43:29.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.083+0000 7ff9ed2f0700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920781 cookie 94657779383040 notifier 4422 bl.length()=159 2024-09-13T23:43:29.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.083+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(67 notify.0 [notify cookie 94657781065984 out=8b] v0'0 uv72 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.083+0000 7ff9ed2f0700 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b5d9c00 con 0x56173b5d9400 2024-09-13T23:43:29.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.083+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.083+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== watch-notify(notify_complete (2) cookie 94657781065984 notify 141733920781 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56173b590b60 con 0x56173b5d9400 2024-09-13T23:43:29.092 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [ 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "foo:foo1", 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: "permissions": "full-control" 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "foo:foo2", 2024-09-13T23:43:29.093 INFO:teuthology.orchestra.run.smithi167.stdout: "permissions": "" 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [ 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo:foo2", 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:29.094 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:29.095 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:29.096 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:29.096 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:29.096 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:29.096 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:29.096 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:29.096 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:29.096 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:29.096 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:29.096 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:29.096 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:29.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.083+0000 7ff9d9ac9700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:29.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.083+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94657781063680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b739c00 con 0x56173b5d9000 2024-09-13T23:43:29.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.087+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94657781063680] v33'83 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.087+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94657779383040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693800 con 0x56173b5d9400 2024-09-13T23:43:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.087+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94657779383040] v33'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.087+0000 7ff9f4342f40 20 remove_watcher() i=0 2024-09-13T23:43:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.087+0000 7ff9f4342f40 2 removed watcher, disabling cache 2024-09-13T23:43:29.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.087+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94657779384192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693400 con 0x56173b5d9000 2024-09-13T23:43:29.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.091+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94657779384192] v33'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b5d9000 2024-09-13T23:43:29.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.091+0000 7ff9f4342f40 20 remove_watcher() i=1 2024-09-13T23:43:29.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.091+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94657779385344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b692c00 con 0x56173b5d9400 2024-09-13T23:43:29.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.095+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94657779385344] v33'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.095+0000 7ff9f4342f40 20 remove_watcher() i=2 2024-09-13T23:43:29.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.095+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94657779386496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b693000 con 0x56173b692400 2024-09-13T23:43:29.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.095+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94657779386496] v33'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.095+0000 7ff9f4342f40 20 remove_watcher() i=3 2024-09-13T23:43:29.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.095+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94657779387648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b739c00 con 0x56173b5d9400 2024-09-13T23:43:29.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.099+0000 7ff9f2afb700 1 -- 172.21.15.167:0/499819871 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94657779387648] v33'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685200 con 0x56173b5d9400 2024-09-13T23:43:29.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.099+0000 7ff9f4342f40 20 remove_watcher() i=4 2024-09-13T23:43:29.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.099+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94657779388800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b739800 con 0x56173b692000 2024-09-13T23:43:29.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.099+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94657779388800] v33'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.099+0000 7ff9f4342f40 20 remove_watcher() i=5 2024-09-13T23:43:29.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.099+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94657779389952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b739400 con 0x56173b692000 2024-09-13T23:43:29.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94657779389952] v33'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b685440 con 0x56173b692000 2024-09-13T23:43:29.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f4342f40 20 remove_watcher() i=6 2024-09-13T23:43:29.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94657779391104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56173b739000 con 0x56173b692400 2024-09-13T23:43:29.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f32fc700 1 -- 172.21.15.167:0/499819871 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94657779391104] v33'74 uv72 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56173b684b40 con 0x56173b692400 2024-09-13T23:43:29.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f4342f40 20 remove_watcher() i=7 2024-09-13T23:43:29.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56173b5d9400 msgr2=0x56173b516100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56173b5d9400 0x56173b516100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56173b5d9000 msgr2=0x56173b515b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56173b5d9000 0x56173b515b80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56173b692400 msgr2=0x56173b516c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56173b692400 0x56173b516c00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56173b692000 msgr2=0x56173b516680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.103+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56173b692000 0x56173b516680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b5d8c00 msgr2=0x56173b515600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b5d8c00 0x56173b515600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x56173b50f920 tx=0x56173b333800 comp rx=0 tx=0).stop 2024-09-13T23:43:29.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 msgr2=0x56173b514b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/499819871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f800 0x56173b514b00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x56173b38bfb0 tx=0x56173b592cc0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 reap_dead start 2024-09-13T23:43:29.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f22fa700 1 -- 172.21.15.167:0/499819871 reap_dead start 2024-09-13T23:43:29.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 shutdown_connections 2024-09-13T23:43:29.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 >> 172.21.15.167:0/499819871 conn(0x56173b22f400 msgr2=0x56173b32f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 shutdown_connections 2024-09-13T23:43:29.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/499819871 wait complete. 2024-09-13T23:43:29.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3981102410 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b738000 msgr2=0x56173b73e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/3981102410 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b738000 0x56173b73e000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x56173b6438c0 tx=0x56173b5937a0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3981102410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 msgr2=0x56173b517180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/3981102410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 0x56173b517180 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x56173b6ae390 tx=0x56173b6b0300 comp rx=0 tx=0).stop 2024-09-13T23:43:29.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3981102410 shutdown_connections 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/3981102410 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b738000 0x56173b73e000 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/3981102410 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b6a0c00 0x56173b517180 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3981102410 >> 172.21.15.167:0/3981102410 conn(0x56173b6a1c00 msgr2=0x56173b330000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3981102410 shutdown_connections 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/3981102410 wait complete. 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/2947051324 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b5d8000 msgr2=0x56173b515080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/2947051324 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b5d8000 0x56173b515080 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x56173b55af60 tx=0x56173b592240 comp rx=0 tx=0).stop 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/2947051324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 msgr2=0x56173b514580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/2947051324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 0x56173b514580 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x56173b38bf20 tx=0x56173b333a40 comp rx=0 tx=0).stop 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/2947051324 shutdown_connections 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/2947051324 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56173b5d8000 0x56173b515080 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 --2- 172.21.15.167:0/2947051324 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56173b22f000 0x56173b514580 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/2947051324 >> 172.21.15.167:0/2947051324 conn(0x56173b22ec00 msgr2=0x56173b32f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/2947051324 shutdown_connections 2024-09-13T23:43:29.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.107+0000 7ff9f4342f40 1 -- 172.21.15.167:0/2947051324 wait complete. 2024-09-13T23:43:29.122 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:29.122 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo1'] 2024-09-13T23:43:29.122 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo1'] 2024-09-13T23:43:29.123 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo1 2024-09-13T23:43:29.163 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:29.164 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:29.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.155+0000 7f8d64576f40 1 Processor -- start 2024-09-13T23:43:29.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.155+0000 7f8d64576f40 1 -- start start 2024-09-13T23:43:29.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.155+0000 7f8d64576f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x561e5633d000 0x561e56622580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.155+0000 7f8d64576f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561e56401500 con 0x561e5633d400 2024-09-13T23:43:29.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.155+0000 7f8d64576f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x561e563caa00 con 0x561e5633d000 2024-09-13T23:43:29.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.155+0000 7f8d62d2f700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x561e5633d000 0x561e56622580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.155+0000 7f8d6252e700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x561e5633d400 0x561e5643c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45358/0 (socket says 172.21.15.167:45358) 2024-09-13T23:43:29.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.155+0000 7f8d6252e700 1 -- 172.21.15.167:0/1953188874 learned_addr learned my addr 172.21.15.167:0/1953188874 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:29.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.155+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1953188874 >> v1:172.21.15.12:6789/0 conn(0x561e5633d400 legacy=0x561e5643c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.155+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1953188874 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e5634fd40 con 0x561e5633d000 2024-09-13T23:43:29.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d62d2f700 1 --2- 172.21.15.167:0/1953188874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d000 0x561e56622580 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x561e5661da70 tx=0x561e56441920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257c6cab8193d5d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d61d2d700 1 -- 172.21.15.167:0/1953188874 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e563e14a0 con 0x561e5633d000 2024-09-13T23:43:29.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d61d2d700 1 -- 172.21.15.167:0/1953188874 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e56401800 con 0x561e5633d000 2024-09-13T23:43:29.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d61d2d700 1 -- 172.21.15.167:0/1953188874 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e563e1600 con 0x561e5633d000 2024-09-13T23:43:29.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d64576f40 1 -- 172.21.15.167:0/1953188874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d000 msgr2=0x561e56622580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1953188874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d000 0x561e56622580 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x561e5661da70 tx=0x561e56441920 comp rx=0 tx=0).stop 2024-09-13T23:43:29.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d64576f40 1 -- 172.21.15.167:0/1953188874 shutdown_connections 2024-09-13T23:43:29.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1953188874 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d000 0x561e56622580 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d64576f40 1 -- 172.21.15.167:0/1953188874 >> 172.21.15.167:0/1953188874 conn(0x561e5633cc00 msgr2=0x561e5643c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d64576f40 1 -- 172.21.15.167:0/1953188874 shutdown_connections 2024-09-13T23:43:29.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d64576f40 1 -- 172.21.15.167:0/1953188874 wait complete. 2024-09-13T23:43:29.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d64576f40 1 Processor -- start 2024-09-13T23:43:29.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d64576f40 1 -- start start 2024-09-13T23:43:29.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d64576f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 0x561e56622b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d64576f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e563ca500 con 0x561e5633d400 2024-09-13T23:43:29.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d62d2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 0x561e56622b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d62d2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 0x561e56622b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51464/0 (socket says 172.21.15.167:51464) 2024-09-13T23:43:29.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d62d2f700 1 -- 172.21.15.167:0/4156054133 learned_addr learned my addr 172.21.15.167:0/4156054133 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d62d2f700 1 -- 172.21.15.167:0/4156054133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e5634fba0 con 0x561e5633d400 2024-09-13T23:43:29.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d62d2f700 1 --2- 172.21.15.167:0/4156054133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 0x561e56622b00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x561e571a18c0 tx=0x561e56441800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0000569e2ca96f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d6152c700 1 -- 172.21.15.167:0/4156054133 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e563e1760 con 0x561e5633d400 2024-09-13T23:43:29.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d6152c700 1 -- 172.21.15.167:0/4156054133 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e56401b00 con 0x561e5633d400 2024-09-13T23:43:29.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.159+0000 7f8d6152c700 1 -- 172.21.15.167:0/4156054133 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e563e14a0 con 0x561e5633d400 2024-09-13T23:43:29.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 -- 172.21.15.167:0/4156054133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 msgr2=0x561e56622b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 --2- 172.21.15.167:0/4156054133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 0x561e56622b00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x561e571a18c0 tx=0x561e56441800 comp rx=0 tx=0).stop 2024-09-13T23:43:29.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 -- 172.21.15.167:0/4156054133 shutdown_connections 2024-09-13T23:43:29.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 --2- 172.21.15.167:0/4156054133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 0x561e56622b00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 -- 172.21.15.167:0/4156054133 >> 172.21.15.167:0/4156054133 conn(0x561e5633cc00 msgr2=0x561e5643d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 -- 172.21.15.167:0/4156054133 shutdown_connections 2024-09-13T23:43:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 -- 172.21.15.167:0/4156054133 wait complete. 2024-09-13T23:43:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 Processor -- start 2024-09-13T23:43:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 -- start start 2024-09-13T23:43:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 0x561e56622b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e563ca3c0 con 0x561e5633d400 2024-09-13T23:43:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d62d2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 0x561e56622b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d62d2f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 0x561e56622b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51470/0 (socket says 172.21.15.167:51470) 2024-09-13T23:43:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d62d2f700 1 -- 172.21.15.167:0/339733236 learned_addr learned my addr 172.21.15.167:0/339733236 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d62d2f700 1 -- 172.21.15.167:0/339733236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e5634fa00 con 0x561e5633d400 2024-09-13T23:43:29.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d62d2f700 1 --2- 172.21.15.167:0/339733236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 0x561e56622b00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x561e571a1f80 tx=0x561e56441920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d5fd29700 1 -- 172.21.15.167:0/339733236 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e563e1600 con 0x561e5633d400 2024-09-13T23:43:29.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d5fd29700 1 -- 172.21.15.167:0/339733236 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e56401c80 con 0x561e5633d400 2024-09-13T23:43:29.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 -- 172.21.15.167:0/339733236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561e5634f860 con 0x561e5633d400 2024-09-13T23:43:29.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d5fd29700 1 -- 172.21.15.167:0/339733236 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e563e18c0 con 0x561e5633d400 2024-09-13T23:43:29.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.163+0000 7f8d64576f40 1 -- 172.21.15.167:0/339733236 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e5634f6c0 con 0x561e5633d400 2024-09-13T23:43:29.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d64576f40 1 Processor -- start 2024-09-13T23:43:29.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d64576f40 1 -- start start 2024-09-13T23:43:29.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d5fd29700 1 -- 172.21.15.167:0/339733236 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x561e56436b00 con 0x561e5633d400 2024-09-13T23:43:29.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d64576f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 0x561e56623080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d64576f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e56362640 con 0x561e5633dc00 2024-09-13T23:43:29.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d63530700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 0x561e56623080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d5fd29700 1 --2- 172.21.15.167:0/339733236 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e566de000 0x561e56624100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d63530700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 0x561e56623080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51472/0 (socket says 172.21.15.167:51472) 2024-09-13T23:43:29.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d63530700 1 -- 172.21.15.167:0/3585582047 learned_addr learned my addr 172.21.15.167:0/3585582047 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d5fd29700 1 -- 172.21.15.167:0/339733236 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x561e563a3180 con 0x561e5633d400 2024-09-13T23:43:29.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d62d2f700 1 --2- 172.21.15.167:0/339733236 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e566de000 0x561e56624100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d63530700 1 -- 172.21.15.167:0/3585582047 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e5634e000 con 0x561e5633dc00 2024-09-13T23:43:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d62d2f700 1 --2- 172.21.15.167:0/339733236 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e566de000 0x561e56624100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x561e571b1020 tx=0x561e566a0c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d63530700 1 --2- 172.21.15.167:0/3585582047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 0x561e56623080 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x561e571b01e0 tx=0x561e566a0b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12454ded3ee9638c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d5d524700 1 -- 172.21.15.167:0/3585582047 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e563e1600 con 0x561e5633dc00 2024-09-13T23:43:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d5d524700 1 -- 172.21.15.167:0/3585582047 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e566fa000 con 0x561e5633dc00 2024-09-13T23:43:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d5d524700 1 -- 172.21.15.167:0/3585582047 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e563e18c0 con 0x561e5633dc00 2024-09-13T23:43:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d64576f40 1 -- 172.21.15.167:0/3585582047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 msgr2=0x561e56623080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d64576f40 1 --2- 172.21.15.167:0/3585582047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 0x561e56623080 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x561e571b01e0 tx=0x561e566a0b40 comp rx=0 tx=0).stop 2024-09-13T23:43:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d64576f40 1 -- 172.21.15.167:0/3585582047 shutdown_connections 2024-09-13T23:43:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d64576f40 1 --2- 172.21.15.167:0/3585582047 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 0x561e56623080 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d64576f40 1 -- 172.21.15.167:0/3585582047 >> 172.21.15.167:0/3585582047 conn(0x561e5633d800 msgr2=0x561e5643d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d64576f40 1 -- 172.21.15.167:0/3585582047 shutdown_connections 2024-09-13T23:43:29.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.167+0000 7f8d64576f40 1 -- 172.21.15.167:0/3585582047 wait complete. 2024-09-13T23:43:29.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d64576f40 1 Processor -- start 2024-09-13T23:43:29.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d64576f40 1 -- start start 2024-09-13T23:43:29.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d64576f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 0x561e56623080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d64576f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e563628c0 con 0x561e5633dc00 2024-09-13T23:43:29.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d63530700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 0x561e56623080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d63530700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 0x561e56623080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51482/0 (socket says 172.21.15.167:51482) 2024-09-13T23:43:29.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 learned_addr learned my addr 172.21.15.167:0/1841983663 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e5634eea0 con 0x561e5633dc00 2024-09-13T23:43:29.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d63530700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 0x561e56623080 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x561e571b0960 tx=0x561e566a05a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d5bd21700 1 -- 172.21.15.167:0/1841983663 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e563e1760 con 0x561e5633dc00 2024-09-13T23:43:29.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d5bd21700 1 -- 172.21.15.167:0/1841983663 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e566fa180 con 0x561e5633dc00 2024-09-13T23:43:29.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d5bd21700 1 -- 172.21.15.167:0/1841983663 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e563e1600 con 0x561e5633dc00 2024-09-13T23:43:29.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561e5634e820 con 0x561e5633dc00 2024-09-13T23:43:29.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.171+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e5634e9c0 con 0x561e5633dc00 2024-09-13T23:43:29.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d5bd21700 1 -- 172.21.15.167:0/1841983663 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x561e56436dc0 con 0x561e5633dc00 2024-09-13T23:43:29.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d5bd21700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e566dec00 0x561e56624680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d5bd21700 1 -- 172.21.15.167:0/1841983663 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x561e563a3340 con 0x561e5633dc00 2024-09-13T23:43:29.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d6252e700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e566dec00 0x561e56624680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561e566df000 0x561e56623600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e566df400 con 0x561e566df000 2024-09-13T23:43:29.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d63530700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561e566df000 0x561e56623600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d6252e700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e566dec00 0x561e56624680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x561e566fdda0 tx=0x561e567980c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d63530700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561e566df000 0x561e56623600 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 20 realm 2024-09-13T23:43:29.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e566df800 con 0x561e566df000 2024-09-13T23:43:29.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:29.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e566dfc00 con 0x561e566df000 2024-09-13T23:43:29.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561e566df400 0x561e56623b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e566df800 con 0x561e566df400 2024-09-13T23:43:29.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.175+0000 7f8d62d2f700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561e566df400 0x561e56623b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d62d2f700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561e566df400 0x561e56623b80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d64576f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:29.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d64576f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e566dfc00 con 0x561e566df400 2024-09-13T23:43:29.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d64576f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:29.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d64576f40 20 searching for the correct realm 2024-09-13T23:43:29.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561e566df800 0x561e56624c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e566dfc00 con 0x561e566df800 2024-09-13T23:43:29.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d6252e700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561e566df800 0x561e56624c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d6252e700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561e566df800 0x561e56624c00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.179+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e566dfc00 con 0x561e566df400 2024-09-13T23:43:29.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d62d2f700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561e566dfc00 0x561e56622580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566dfc00 2024-09-13T23:43:29.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d63530700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561e566dfc00 0x561e56622580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d63530700 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561e566dfc00 0x561e56622580 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df000 2024-09-13T23:43:29.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df000 2024-09-13T23:43:29.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df800 2024-09-13T23:43:29.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df000 2024-09-13T23:43:29.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.183+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df400 2024-09-13T23:43:29.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df400 2024-09-13T23:43:29.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df000 2024-09-13T23:43:29.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566dfc00 2024-09-13T23:43:29.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df000 2024-09-13T23:43:29.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566dfc00 2024-09-13T23:43:29.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.187+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df400 2024-09-13T23:43:29.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df800 2024-09-13T23:43:29.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df000 2024-09-13T23:43:29.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df400 2024-09-13T23:43:29.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df800 2024-09-13T23:43:29.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df000 2024-09-13T23:43:29.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df000 2024-09-13T23:43:29.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.191+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df000 2024-09-13T23:43:29.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df800 2024-09-13T23:43:29.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566dfc00 2024-09-13T23:43:29.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df000 2024-09-13T23:43:29.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566dfc00 2024-09-13T23:43:29.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df800 2024-09-13T23:43:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566dfc00 2024-09-13T23:43:29.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.195+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df400 2024-09-13T23:43:29.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df400 2024-09-13T23:43:29.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df800 2024-09-13T23:43:29.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df400 2024-09-13T23:43:29.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df400 2024-09-13T23:43:29.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d64576f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:29.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d64576f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:29.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d64576f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:29.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d64576f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:29.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d64576f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac800 con 0x561e566df000 2024-09-13T23:43:29.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d64576f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d64576f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.199+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567acc00 con 0x561e566df400 2024-09-13T23:43:29.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:29.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ad000 con 0x561e566df400 2024-09-13T23:43:29.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:29.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 zone default found 2024-09-13T23:43:29.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 4 Realm: () 2024-09-13T23:43:29.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:29.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:29.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:29.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 zonegroup default 2024-09-13T23:43:29.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ad400 con 0x561e566dfc00 2024-09-13T23:43:29.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ad800 con 0x561e566df800 2024-09-13T23:43:29.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567adc00 con 0x561e566df800 2024-09-13T23:43:29.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 started sync module instance, tier type = 2024-09-13T23:43:29.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:29.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.203+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ad800 con 0x561e566df400 2024-09-13T23:43:29.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.207+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.207+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94688298244864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567adc00 con 0x561e566df400 2024-09-13T23:43:29.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.207+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac400 con 0x561e566df000 2024-09-13T23:43:29.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.211+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94688298244864] v33'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.211+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.211+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94688298246016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df000 2024-09-13T23:43:29.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.211+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567acc00 con 0x561e566df400 2024-09-13T23:43:29.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.211+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94688298246016] v33'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.215+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.215+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94688298247168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ad000 con 0x561e566df400 2024-09-13T23:43:29.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.215+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac800 con 0x561e566dfc00 2024-09-13T23:43:29.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.215+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94688298247168] v33'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.215+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.215+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94688298248320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ad400 con 0x561e566dfc00 2024-09-13T23:43:29.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.215+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac400 con 0x561e566df400 2024-09-13T23:43:29.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.219+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94688298248320] v33'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.219+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.219+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94688298249472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac000 con 0x561e566df400 2024-09-13T23:43:29.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.219+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ad800 con 0x561e566df800 2024-09-13T23:43:29.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.219+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.219+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94688298250624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567adc00 con 0x561e566df800 2024-09-13T23:43:29.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.219+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567acc00 con 0x561e566df800 2024-09-13T23:43:29.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.219+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94688298249472] v33'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.223+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.223+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94688298251776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ad000 con 0x561e566df800 2024-09-13T23:43:29.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.223+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ac400 con 0x561e566dfc00 2024-09-13T23:43:29.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.223+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94688298250624] v33'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.223+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94688298251776] v33'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.223+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'75 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94688298252928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567ad800 con 0x561e566dfc00 2024-09-13T23:43:29.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d64576f40 20 add_watcher() i=0 2024-09-13T23:43:29.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d64576f40 20 add_watcher() i=1 2024-09-13T23:43:29.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d64576f40 20 add_watcher() i=2 2024-09-13T23:43:29.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d64576f40 20 add_watcher() i=3 2024-09-13T23:43:29.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d64576f40 20 add_watcher() i=4 2024-09-13T23:43:29.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d64576f40 20 add_watcher() i=5 2024-09-13T23:43:29.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d64576f40 20 add_watcher() i=6 2024-09-13T23:43:29.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94688298252928] v33'76 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d64576f40 20 add_watcher() i=7 2024-09-13T23:43:29.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d64576f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:29.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567adc00 con 0x561e566df000 2024-09-13T23:43:29.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.227+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94688299950080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567acc00 con 0x561e566df000 2024-09-13T23:43:29.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94688299950080] v33'84 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d49cfd700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:29.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d64576f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:29.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d64576f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d64576f40 1 Processor -- start 2024-09-13T23:43:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d64576f40 1 -- start start 2024-09-13T23:43:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d64576f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 0x561e56625180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d64576f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e563ca3c0 con 0x561e567ad400 2024-09-13T23:43:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d6252e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 0x561e56625180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d6252e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 0x561e56625180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51496/0 (socket says 172.21.15.167:51496) 2024-09-13T23:43:29.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d6252e700 1 -- 172.21.15.167:0/3538820837 learned_addr learned my addr 172.21.15.167:0/3538820837 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d6252e700 1 -- 172.21.15.167:0/3538820837 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e5634eb60 con 0x561e567ad400 2024-09-13T23:43:29.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d6252e700 1 --2- 172.21.15.167:0/3538820837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 0x561e56625180 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x561e567c4090 tx=0x561e56798060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e464771f6e03085e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d494fc700 1 -- 172.21.15.167:0/3538820837 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e567cc000 con 0x561e567ad400 2024-09-13T23:43:29.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d494fc700 1 -- 172.21.15.167:0/3538820837 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e566fa600 con 0x561e567ad400 2024-09-13T23:43:29.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.231+0000 7f8d494fc700 1 -- 172.21.15.167:0/3538820837 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e567cc160 con 0x561e567ad400 2024-09-13T23:43:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 -- 172.21.15.167:0/3538820837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 msgr2=0x561e56625180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 --2- 172.21.15.167:0/3538820837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 0x561e56625180 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x561e567c4090 tx=0x561e56798060 comp rx=0 tx=0).stop 2024-09-13T23:43:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 -- 172.21.15.167:0/3538820837 shutdown_connections 2024-09-13T23:43:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 --2- 172.21.15.167:0/3538820837 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 0x561e56625180 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 -- 172.21.15.167:0/3538820837 >> 172.21.15.167:0/3538820837 conn(0x561e567ac800 msgr2=0x561e5643e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 -- 172.21.15.167:0/3538820837 shutdown_connections 2024-09-13T23:43:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 -- 172.21.15.167:0/3538820837 wait complete. 2024-09-13T23:43:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 Processor -- start 2024-09-13T23:43:29.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 -- start start 2024-09-13T23:43:29.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 0x561e56625180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e563ca500 con 0x561e567ad400 2024-09-13T23:43:29.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d6252e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 0x561e56625180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d6252e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 0x561e56625180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51498/0 (socket says 172.21.15.167:51498) 2024-09-13T23:43:29.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d6252e700 1 -- 172.21.15.167:0/1379107586 learned_addr learned my addr 172.21.15.167:0/1379107586 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d6252e700 1 -- 172.21.15.167:0/1379107586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e5634ed00 con 0x561e567ad400 2024-09-13T23:43:29.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d6252e700 1 --2- 172.21.15.167:0/1379107586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 0x561e56625180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x561e567c5350 tx=0x561e56798180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d47cf9700 1 -- 172.21.15.167:0/1379107586 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e567cc2c0 con 0x561e567ad400 2024-09-13T23:43:29.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d47cf9700 1 -- 172.21.15.167:0/1379107586 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e566fa300 con 0x561e567ad400 2024-09-13T23:43:29.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d47cf9700 1 -- 172.21.15.167:0/1379107586 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561e567cc000 con 0x561e567ad400 2024-09-13T23:43:29.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 -- 172.21.15.167:0/1379107586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561e5634e340 con 0x561e567ad400 2024-09-13T23:43:29.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.235+0000 7f8d64576f40 1 -- 172.21.15.167:0/1379107586 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e5634eb60 con 0x561e567ad400 2024-09-13T23:43:29.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d47cf9700 1 -- 172.21.15.167:0/1379107586 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x561e56437b80 con 0x561e567ad400 2024-09-13T23:43:29.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d47cf9700 1 --2- 172.21.15.167:0/1379107586 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e56844000 0x561e5684a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d47cf9700 1 -- 172.21.15.167:0/1379107586 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x561e563a3500 con 0x561e567ad400 2024-09-13T23:43:29.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d64576f40 5 note: GC not initialized 2024-09-13T23:43:29.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d63530700 1 --2- 172.21.15.167:0/1379107586 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e56844000 0x561e5684a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d63530700 1 --2- 172.21.15.167:0/1379107586 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e56844000 0x561e5684a000 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x561e56743b90 tx=0x561e566a1500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d454f4700 20 reqs_thread_entry: start 2024-09-13T23:43:29.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d64576f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:29.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d64576f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:29.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d434f0700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:59 2024 2024-09-13T23:43:29.250 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:29.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d42cce700 20 reqs_thread_entry: start 2024-09-13T23:43:29.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d64576f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:29.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d64576f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.239+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e567adc00 con 0x561e566df000 2024-09-13T23:43:29.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.243+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~539 out=539b,getxattrs out=75b] v0'0 uv9 ondisk = 0) v8 ==== 231+0+662 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.243+0000 7f8d64576f40 20 rados_obj.operate() r=0 bl.length=539 2024-09-13T23:43:29.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.243+0000 7f8d64576f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:29.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.243+0000 7f8d64576f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:29.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.243+0000 7f8d64576f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:29.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.243+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~513 in=513b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e56844800 con 0x561e566df000 2024-09-13T23:43:29.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.243+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~513] v33'10 uv10 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.243+0000 7f8d64576f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:29.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.243+0000 7f8d64576f40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:29.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.243+0000 7f8d64576f40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:29.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.243+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94688299952384 in=701b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e56844c00 con 0x561e566df400 2024-09-13T23:43:29.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.247+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94688298249472 notify 141733920782 ret 0) v3 ==== 731+0+0 (crc 0 0 0) 0x561e56694680 con 0x561e566df400 2024-09-13T23:43:29.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.247+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(64 notify.4 [notify cookie 94688299952384 out=8b] v0'0 uv75 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.247+0000 7f8d5d524700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920782 cookie 94688298249472 notifier 4430 bl.length()=689 2024-09-13T23:43:29.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.247+0000 7f8d5d524700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:29.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.247+0000 7f8d5d524700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:29.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.247+0000 7f8d5d524700 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e568b0000 con 0x561e566df400 2024-09-13T23:43:29.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.247+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.247+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94688299952384 notify 141733920782 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x561e56694820 con 0x561e566df400 2024-09-13T23:43:29.256 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:29.256 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:29.256 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:29.256 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:29.256 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:29.256 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:29.256 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [ 2024-09-13T23:43:29.256 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:29.256 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "foo:foo2", 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: "permissions": "" 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [ 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:29.257 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo:foo2", 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy" 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:29.258 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:29.259 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:29.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.247+0000 7f8d49cfd700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:29.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.247+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94688299950080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e56845000 con 0x561e566df000 2024-09-13T23:43:29.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.251+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94688299950080] v33'85 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.251+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94688298244864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e56845400 con 0x561e566df400 2024-09-13T23:43:29.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.251+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94688298244864] v33'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.251+0000 7f8d64576f40 20 remove_watcher() i=0 2024-09-13T23:43:29.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.251+0000 7f8d64576f40 2 removed watcher, disabling cache 2024-09-13T23:43:29.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.251+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94688298246016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e56845800 con 0x561e566df000 2024-09-13T23:43:29.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.255+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94688298246016] v33'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566df000 2024-09-13T23:43:29.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.255+0000 7f8d64576f40 20 remove_watcher() i=1 2024-09-13T23:43:29.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.255+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94688298247168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e56845c00 con 0x561e566df400 2024-09-13T23:43:29.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.259+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94688298247168] v33'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.259+0000 7f8d64576f40 20 remove_watcher() i=2 2024-09-13T23:43:29.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.259+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94688298248320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e568b0800 con 0x561e566dfc00 2024-09-13T23:43:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.259+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94688298248320] v33'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.259+0000 7f8d64576f40 20 remove_watcher() i=3 2024-09-13T23:43:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.259+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94688298249472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e568b0c00 con 0x561e566df400 2024-09-13T23:43:29.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.263+0000 7f8d62d2f700 1 -- 172.21.15.167:0/1841983663 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94688298249472] v33'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787200 con 0x561e566df400 2024-09-13T23:43:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.263+0000 7f8d64576f40 20 remove_watcher() i=4 2024-09-13T23:43:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.263+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94688298250624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e568b1000 con 0x561e566df800 2024-09-13T23:43:29.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.267+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94688298250624] v33'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.267+0000 7f8d64576f40 20 remove_watcher() i=5 2024-09-13T23:43:29.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.267+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94688298251776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e568b1400 con 0x561e566df800 2024-09-13T23:43:29.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.267+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94688298251776] v33'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56787440 con 0x561e566df800 2024-09-13T23:43:29.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.267+0000 7f8d64576f40 20 remove_watcher() i=6 2024-09-13T23:43:29.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.267+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94688298252928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561e568b1800 con 0x561e566dfc00 2024-09-13T23:43:29.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d63530700 1 -- 172.21.15.167:0/1841983663 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94688298252928] v33'77 uv75 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561e56786b40 con 0x561e566dfc00 2024-09-13T23:43:29.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 20 remove_watcher() i=7 2024-09-13T23:43:29.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561e566df400 msgr2=0x561e56623b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561e566df400 0x561e56623b80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561e566df000 msgr2=0x561e56623600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561e566df000 0x561e56623600 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561e566dfc00 msgr2=0x561e56622580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561e566dfc00 0x561e56622580 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561e566df800 msgr2=0x561e56624c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561e566df800 0x561e56624c00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e566dec00 msgr2=0x561e56624680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e566dec00 0x561e56624680 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x561e566fdda0 tx=0x561e567980c0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 msgr2=0x561e56623080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1841983663 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633dc00 0x561e56623080 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x561e571b0960 tx=0x561e566a05a0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 reap_dead start 2024-09-13T23:43:29.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d6252e700 1 -- 172.21.15.167:0/1841983663 reap_dead start 2024-09-13T23:43:29.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 shutdown_connections 2024-09-13T23:43:29.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 >> 172.21.15.167:0/1841983663 conn(0x561e5633d800 msgr2=0x561e5643d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 shutdown_connections 2024-09-13T23:43:29.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1841983663 wait complete. 2024-09-13T23:43:29.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1379107586 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e56844000 msgr2=0x561e5684a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1379107586 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e56844000 0x561e5684a000 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x561e56743b90 tx=0x561e566a1500 comp rx=0 tx=0).stop 2024-09-13T23:43:29.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1379107586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 msgr2=0x561e56625180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1379107586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 0x561e56625180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x561e567c5350 tx=0x561e56798180 comp rx=0 tx=0).stop 2024-09-13T23:43:29.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1379107586 shutdown_connections 2024-09-13T23:43:29.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1379107586 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e56844000 0x561e5684a000 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/1379107586 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e567ad400 0x561e56625180 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1379107586 >> 172.21.15.167:0/1379107586 conn(0x561e567ac800 msgr2=0x561e5643e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1379107586 shutdown_connections 2024-09-13T23:43:29.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/1379107586 wait complete. 2024-09-13T23:43:29.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/339733236 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e566de000 msgr2=0x561e56624100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/339733236 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e566de000 0x561e56624100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x561e571b1020 tx=0x561e566a0c60 comp rx=0 tx=0).stop 2024-09-13T23:43:29.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/339733236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 msgr2=0x561e56622b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/339733236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 0x561e56622b00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x561e571a1f80 tx=0x561e56441920 comp rx=0 tx=0).stop 2024-09-13T23:43:29.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/339733236 shutdown_connections 2024-09-13T23:43:29.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/339733236 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561e566de000 0x561e56624100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 --2- 172.21.15.167:0/339733236 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561e5633d400 0x561e56622b00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/339733236 >> 172.21.15.167:0/339733236 conn(0x561e5633cc00 msgr2=0x561e5643d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/339733236 shutdown_connections 2024-09-13T23:43:29.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.271+0000 7f8d64576f40 1 -- 172.21.15.167:0/339733236 wait complete. 2024-09-13T23:43:29.287 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:29.287 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-09-13T23:43:29.288 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-09-13T23:43:29.288 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo2 --key-type swift --purge-keys 2024-09-13T23:43:29.332 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.323+0000 7fee60107f40 1 Processor -- start 2024-09-13T23:43:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.323+0000 7fee60107f40 1 -- start start 2024-09-13T23:43:29.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.323+0000 7fee60107f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5610650bd000 0x5610653a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.323+0000 7fee60107f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x561065181500 con 0x5610650bd400 2024-09-13T23:43:29.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.323+0000 7fee60107f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x56106514aa00 con 0x5610650bd000 2024-09-13T23:43:29.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.323+0000 7fee5e8c0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5610650bd000 0x5610653a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.323+0000 7fee5e0bf700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5610650bd400 0x5610651bc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45366/0 (socket says 172.21.15.167:45366) 2024-09-13T23:43:29.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.323+0000 7fee5e8c0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5610650bd000 0x5610653a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51506/0 (socket says 172.21.15.167:51506) 2024-09-13T23:43:29.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.323+0000 7fee5e0bf700 1 -- 172.21.15.167:0/1548192864 learned_addr learned my addr 172.21.15.167:0/1548192864 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:29.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee5e8c0700 1 -- 172.21.15.167:0/1548192864 >> v1:172.21.15.12:6789/0 conn(0x5610650bd400 legacy=0x5610651bc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee5e8c0700 1 -- 172.21.15.167:0/1548192864 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610650cfd40 con 0x5610650bd000 2024-09-13T23:43:29.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee5e8c0700 1 --2- 172.21.15.167:0/1548192864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56106539da70 tx=0x5610651c1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fedcd478ac932d8a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee5d8be700 1 -- 172.21.15.167:0/1548192864 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610651614a0 con 0x5610650bd000 2024-09-13T23:43:29.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee5d8be700 1 -- 172.21.15.167:0/1548192864 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561065181800 con 0x5610650bd000 2024-09-13T23:43:29.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee5d8be700 1 -- 172.21.15.167:0/1548192864 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561065161600 con 0x5610650bd000 2024-09-13T23:43:29.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee60107f40 1 -- 172.21.15.167:0/1548192864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 msgr2=0x5610653a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee60107f40 1 --2- 172.21.15.167:0/1548192864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x56106539da70 tx=0x5610651c1920 comp rx=0 tx=0).stop 2024-09-13T23:43:29.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee60107f40 1 -- 172.21.15.167:0/1548192864 shutdown_connections 2024-09-13T23:43:29.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee60107f40 1 --2- 172.21.15.167:0/1548192864 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee60107f40 1 -- 172.21.15.167:0/1548192864 >> 172.21.15.167:0/1548192864 conn(0x5610650bcc00 msgr2=0x5610651bc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee60107f40 1 -- 172.21.15.167:0/1548192864 shutdown_connections 2024-09-13T23:43:29.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee60107f40 1 -- 172.21.15.167:0/1548192864 wait complete. 2024-09-13T23:43:29.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee60107f40 1 Processor -- start 2024-09-13T23:43:29.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee60107f40 1 -- start start 2024-09-13T23:43:29.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee60107f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee60107f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106514a500 con 0x5610650bd000 2024-09-13T23:43:29.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee5e8c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee5e8c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51518/0 (socket says 172.21.15.167:51518) 2024-09-13T23:43:29.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.327+0000 7fee5e8c0700 1 -- 172.21.15.167:0/96088413 learned_addr learned my addr 172.21.15.167:0/96088413 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee5e8c0700 1 -- 172.21.15.167:0/96088413 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610650cfba0 con 0x5610650bd000 2024-09-13T23:43:29.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee5e8c0700 1 --2- 172.21.15.167:0/96088413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x56106539df80 tx=0x5610651c1800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723eb0451a540060 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee5d0bd700 1 -- 172.21.15.167:0/96088413 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561065161760 con 0x5610650bd000 2024-09-13T23:43:29.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee5d0bd700 1 -- 172.21.15.167:0/96088413 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561065181b00 con 0x5610650bd000 2024-09-13T23:43:29.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee5d0bd700 1 -- 172.21.15.167:0/96088413 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610651614a0 con 0x5610650bd000 2024-09-13T23:43:29.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee60107f40 1 -- 172.21.15.167:0/96088413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 msgr2=0x5610653a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee60107f40 1 --2- 172.21.15.167:0/96088413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x56106539df80 tx=0x5610651c1800 comp rx=0 tx=0).stop 2024-09-13T23:43:29.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee60107f40 1 -- 172.21.15.167:0/96088413 shutdown_connections 2024-09-13T23:43:29.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee60107f40 1 --2- 172.21.15.167:0/96088413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee60107f40 1 -- 172.21.15.167:0/96088413 >> 172.21.15.167:0/96088413 conn(0x5610650bcc00 msgr2=0x5610651bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee60107f40 1 -- 172.21.15.167:0/96088413 shutdown_connections 2024-09-13T23:43:29.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee60107f40 1 -- 172.21.15.167:0/96088413 wait complete. 2024-09-13T23:43:29.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee60107f40 1 Processor -- start 2024-09-13T23:43:29.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee60107f40 1 -- start start 2024-09-13T23:43:29.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee60107f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.331+0000 7fee60107f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106514a3c0 con 0x5610650bd000 2024-09-13T23:43:29.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5e8c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5e8c0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51534/0 (socket says 172.21.15.167:51534) 2024-09-13T23:43:29.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5e8c0700 1 -- 172.21.15.167:0/1472314597 learned_addr learned my addr 172.21.15.167:0/1472314597 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5e8c0700 1 -- 172.21.15.167:0/1472314597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610650cfa00 con 0x5610650bd000 2024-09-13T23:43:29.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5e8c0700 1 --2- 172.21.15.167:0/1472314597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x561065219f80 tx=0x5610651c1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5b8ba700 1 -- 172.21.15.167:0/1472314597 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561065161600 con 0x5610650bd000 2024-09-13T23:43:29.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5b8ba700 1 -- 172.21.15.167:0/1472314597 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561065181c80 con 0x5610650bd000 2024-09-13T23:43:29.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee60107f40 1 -- 172.21.15.167:0/1472314597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5610650cf860 con 0x5610650bd000 2024-09-13T23:43:29.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5b8ba700 1 -- 172.21.15.167:0/1472314597 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610651618c0 con 0x5610650bd000 2024-09-13T23:43:29.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee60107f40 1 -- 172.21.15.167:0/1472314597 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610650cf6c0 con 0x5610650bd000 2024-09-13T23:43:29.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee60107f40 1 Processor -- start 2024-09-13T23:43:29.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee60107f40 1 -- start start 2024-09-13T23:43:29.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee60107f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 0x5610653a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee60107f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610650e2640 con 0x5610650bd800 2024-09-13T23:43:29.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5b8ba700 1 -- 172.21.15.167:0/1472314597 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5610651b6b00 con 0x5610650bd000 2024-09-13T23:43:29.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5f0c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 0x5610653a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5f0c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 0x5610653a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51538/0 (socket says 172.21.15.167:51538) 2024-09-13T23:43:29.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5f0c1700 1 -- 172.21.15.167:0/3566130443 learned_addr learned my addr 172.21.15.167:0/3566130443 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5b8ba700 1 --2- 172.21.15.167:0/1472314597 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561065462000 0x5610653a3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.335+0000 7fee5b8ba700 1 -- 172.21.15.167:0/1472314597 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x561065123180 con 0x5610650bd000 2024-09-13T23:43:29.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee5e8c0700 1 --2- 172.21.15.167:0/1472314597 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561065462000 0x5610653a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee5f0c1700 1 -- 172.21.15.167:0/3566130443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610650ce000 con 0x5610650bd800 2024-09-13T23:43:29.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee5f0c1700 1 --2- 172.21.15.167:0/3566130443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 0x5610653a2b00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x56106543c4b0 tx=0x561065424b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35a2b433c37754c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee5e8c0700 1 --2- 172.21.15.167:0/1472314597 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561065462000 0x5610653a3080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5610653e94d0 tx=0x561065424c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee590b5700 1 -- 172.21.15.167:0/3566130443 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561065161600 con 0x5610650bd800 2024-09-13T23:43:29.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee590b5700 1 -- 172.21.15.167:0/3566130443 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561065466000 con 0x5610650bd800 2024-09-13T23:43:29.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee590b5700 1 -- 172.21.15.167:0/3566130443 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5610651618c0 con 0x5610650bd800 2024-09-13T23:43:29.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee60107f40 1 -- 172.21.15.167:0/3566130443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 msgr2=0x5610653a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee60107f40 1 --2- 172.21.15.167:0/3566130443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 0x5610653a2b00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x56106543c4b0 tx=0x561065424b40 comp rx=0 tx=0).stop 2024-09-13T23:43:29.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee60107f40 1 -- 172.21.15.167:0/3566130443 shutdown_connections 2024-09-13T23:43:29.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee60107f40 1 --2- 172.21.15.167:0/3566130443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 0x5610653a2b00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee60107f40 1 -- 172.21.15.167:0/3566130443 >> 172.21.15.167:0/3566130443 conn(0x5610650bd400 msgr2=0x5610651bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee60107f40 1 -- 172.21.15.167:0/3566130443 shutdown_connections 2024-09-13T23:43:29.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee60107f40 1 -- 172.21.15.167:0/3566130443 wait complete. 2024-09-13T23:43:29.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee60107f40 1 Processor -- start 2024-09-13T23:43:29.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee60107f40 1 -- start start 2024-09-13T23:43:29.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee60107f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 0x5610653a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.339+0000 7fee60107f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610650e28c0 con 0x5610650bd800 2024-09-13T23:43:29.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee5f0c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 0x5610653a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee5f0c1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 0x5610653a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51550/0 (socket says 172.21.15.167:51550) 2024-09-13T23:43:29.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 learned_addr learned my addr 172.21.15.167:0/178873202 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610650ceea0 con 0x5610650bd800 2024-09-13T23:43:29.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee5f0c1700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 0x5610653a2b00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x5610653e8270 tx=0x5610654245a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee578b2700 1 -- 172.21.15.167:0/178873202 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561065161760 con 0x5610650bd800 2024-09-13T23:43:29.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee578b2700 1 -- 172.21.15.167:0/178873202 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561065466180 con 0x5610650bd800 2024-09-13T23:43:29.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5610650ce820 con 0x5610650bd800 2024-09-13T23:43:29.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee578b2700 1 -- 172.21.15.167:0/178873202 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x561065161600 con 0x5610650bd800 2024-09-13T23:43:29.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610650ce9c0 con 0x5610650bd800 2024-09-13T23:43:29.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee578b2700 1 -- 172.21.15.167:0/178873202 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5610651b7080 con 0x5610650bd800 2024-09-13T23:43:29.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee60107f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee578b2700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561065462c00 0x5610653a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.343+0000 7fee578b2700 1 -- 172.21.15.167:0/178873202 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x561065123340 con 0x5610650bd800 2024-09-13T23:43:29.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee5e0bf700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561065462c00 0x5610653a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561065463000 0x5610653a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463400 con 0x561065463000 2024-09-13T23:43:29.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee5f0c1700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561065463000 0x5610653a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee5e0bf700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561065462c00 0x5610653a3600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x56106543dd10 tx=0x5610654fa0c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee5f0c1700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561065463000 0x5610653a3b80 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 20 realm 2024-09-13T23:43:29.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463000 2024-09-13T23:43:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463c00 con 0x561065463000 2024-09-13T23:43:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561065463400 0x5610653a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463400 2024-09-13T23:43:29.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.347+0000 7fee5e8c0700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561065463400 0x5610653a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee5e8c0700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561065463400 0x5610653a4100 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee60107f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:29.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee60107f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463c00 con 0x561065463400 2024-09-13T23:43:29.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee60107f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:29.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee60107f40 20 searching for the correct realm 2024-09-13T23:43:29.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee60107f40 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561065522000 0x5610653a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065522400 con 0x561065522000 2024-09-13T23:43:29.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee5e0bf700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561065522000 0x5610653a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee5e0bf700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561065522000 0x5610653a4680 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065522400 con 0x561065463400 2024-09-13T23:43:29.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee5e8c0700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561065522400 0x5610653a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522400 2024-09-13T23:43:29.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.351+0000 7fee5f0c1700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561065522400 0x5610653a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5f0c1700 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561065522400 0x5610653a4c00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463000 2024-09-13T23:43:29.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463000 2024-09-13T23:43:29.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522000 2024-09-13T23:43:29.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463000 2024-09-13T23:43:29.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463400 2024-09-13T23:43:29.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.355+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463400 2024-09-13T23:43:29.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463000 2024-09-13T23:43:29.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522400 2024-09-13T23:43:29.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463000 2024-09-13T23:43:29.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522400 2024-09-13T23:43:29.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463400 2024-09-13T23:43:29.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522000 2024-09-13T23:43:29.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.359+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463000 2024-09-13T23:43:29.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463400 2024-09-13T23:43:29.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522000 2024-09-13T23:43:29.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463000 2024-09-13T23:43:29.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463000 2024-09-13T23:43:29.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463000 2024-09-13T23:43:29.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522000 2024-09-13T23:43:29.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.363+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522400 2024-09-13T23:43:29.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463000 2024-09-13T23:43:29.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522400 2024-09-13T23:43:29.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522000 2024-09-13T23:43:29.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522400 2024-09-13T23:43:29.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463400 2024-09-13T23:43:29.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463400 2024-09-13T23:43:29.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.367+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522000 2024-09-13T23:43:29.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463400 2024-09-13T23:43:29.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065463400 2024-09-13T23:43:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee60107f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee60107f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee60107f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee60107f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee60107f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065522800 con 0x561065463000 2024-09-13T23:43:29.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee60107f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee60107f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065522c00 con 0x561065463400 2024-09-13T23:43:29.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee60107f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:29.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee60107f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.371+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523000 con 0x561065463400 2024-09-13T23:43:29.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 20 zone default found 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 4 Realm: () 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 20 zonegroup default 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523400 con 0x561065522400 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523800 con 0x561065522000 2024-09-13T23:43:29.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523c00 con 0x561065522000 2024-09-13T23:43:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 20 started sync module instance, tier type = 2024-09-13T23:43:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.375+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523800 con 0x561065463400 2024-09-13T23:43:29.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.379+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.379+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94628417739520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523c00 con 0x561065463400 2024-09-13T23:43:29.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.379+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065522c00 con 0x561065463000 2024-09-13T23:43:29.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.383+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.383+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94628417740672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523000 con 0x561065463000 2024-09-13T23:43:29.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.383+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94628417739520] v33'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.383+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523800 con 0x561065463400 2024-09-13T23:43:29.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.383+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94628417740672] v33'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.383+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.383+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94628417741824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463c00 con 0x561065463400 2024-09-13T23:43:29.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.383+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522400 2024-09-13T23:43:29.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.387+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.387+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94628417741824] v33'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.387+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94628417742976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065522800 con 0x561065522400 2024-09-13T23:43:29.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.387+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523400 con 0x561065463400 2024-09-13T23:43:29.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.391+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94628417742976] v33'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.391+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.391+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94628417744128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065522c00 con 0x561065463400 2024-09-13T23:43:29.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.391+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523000 con 0x561065522000 2024-09-13T23:43:29.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.391+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94628417744128] v33'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.391+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.391+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94628417745280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065528000 con 0x561065522000 2024-09-13T23:43:29.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.391+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065528400 con 0x561065522000 2024-09-13T23:43:29.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94628417745280] v33'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94628417746432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065528800 con 0x561065522000 2024-09-13T23:43:29.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065528c00 con 0x561065522400 2024-09-13T23:43:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'78 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94628417747584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065529000 con 0x561065522400 2024-09-13T23:43:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee60107f40 20 add_watcher() i=0 2024-09-13T23:43:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee60107f40 20 add_watcher() i=1 2024-09-13T23:43:29.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee60107f40 20 add_watcher() i=2 2024-09-13T23:43:29.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee60107f40 20 add_watcher() i=3 2024-09-13T23:43:29.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee60107f40 20 add_watcher() i=4 2024-09-13T23:43:29.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee60107f40 20 add_watcher() i=5 2024-09-13T23:43:29.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94628417746432] v33'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.395+0000 7fee60107f40 20 add_watcher() i=6 2024-09-13T23:43:29.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.399+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94628417747584] v33'79 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.399+0000 7fee60107f40 20 add_watcher() i=7 2024-09-13T23:43:29.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.399+0000 7fee60107f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:29.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.399+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065529400 con 0x561065463000 2024-09-13T23:43:29.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.399+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.399+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94628419395584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065529800 con 0x561065463000 2024-09-13T23:43:29.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94628419395584] v33'86 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee4588e700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:29.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee60107f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:29.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee60107f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:29.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee60107f40 1 Processor -- start 2024-09-13T23:43:29.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee60107f40 1 -- start start 2024-09-13T23:43:29.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee60107f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 0x5610653a5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee60107f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106514a3c0 con 0x561065528c00 2024-09-13T23:43:29.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee5e0bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 0x5610653a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee5e0bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 0x5610653a5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51552/0 (socket says 172.21.15.167:51552) 2024-09-13T23:43:29.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee5e0bf700 1 -- 172.21.15.167:0/1302229961 learned_addr learned my addr 172.21.15.167:0/1302229961 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee5e0bf700 1 -- 172.21.15.167:0/1302229961 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610650ceb60 con 0x561065528c00 2024-09-13T23:43:29.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee5e0bf700 1 --2- 172.21.15.167:0/1302229961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 0x5610653a5180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x561065536090 tx=0x5610654fa060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc29c238e278b163 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee4508d700 1 -- 172.21.15.167:0/1302229961 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56106553e000 con 0x561065528c00 2024-09-13T23:43:29.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee4508d700 1 -- 172.21.15.167:0/1302229961 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561065466600 con 0x561065528c00 2024-09-13T23:43:29.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.403+0000 7fee4508d700 1 -- 172.21.15.167:0/1302229961 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56106553e160 con 0x561065528c00 2024-09-13T23:43:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 -- 172.21.15.167:0/1302229961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 msgr2=0x5610653a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 --2- 172.21.15.167:0/1302229961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 0x5610653a5180 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x561065536090 tx=0x5610654fa060 comp rx=0 tx=0).stop 2024-09-13T23:43:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 -- 172.21.15.167:0/1302229961 shutdown_connections 2024-09-13T23:43:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 --2- 172.21.15.167:0/1302229961 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 0x5610653a5180 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 -- 172.21.15.167:0/1302229961 >> 172.21.15.167:0/1302229961 conn(0x561065529c00 msgr2=0x5610651be000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 -- 172.21.15.167:0/1302229961 shutdown_connections 2024-09-13T23:43:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 -- 172.21.15.167:0/1302229961 wait complete. 2024-09-13T23:43:29.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 Processor -- start 2024-09-13T23:43:29.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 -- start start 2024-09-13T23:43:29.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 0x5610653a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56106514a500 con 0x561065528c00 2024-09-13T23:43:29.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee5e0bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 0x5610653a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee5e0bf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 0x5610653a5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51562/0 (socket says 172.21.15.167:51562) 2024-09-13T23:43:29.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee5e0bf700 1 -- 172.21.15.167:0/1974290650 learned_addr learned my addr 172.21.15.167:0/1974290650 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee5e0bf700 1 -- 172.21.15.167:0/1974290650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610650ced00 con 0x561065528c00 2024-09-13T23:43:29.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee5e0bf700 1 --2- 172.21.15.167:0/1974290650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 0x5610653a5180 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x561065537350 tx=0x5610654fa180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee4388a700 1 -- 172.21.15.167:0/1974290650 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56106553e2c0 con 0x561065528c00 2024-09-13T23:43:29.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee4388a700 1 -- 172.21.15.167:0/1974290650 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561065466300 con 0x561065528c00 2024-09-13T23:43:29.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee4388a700 1 -- 172.21.15.167:0/1974290650 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56106553e000 con 0x561065528c00 2024-09-13T23:43:29.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 -- 172.21.15.167:0/1974290650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5610650ce340 con 0x561065528c00 2024-09-13T23:43:29.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.407+0000 7fee60107f40 1 -- 172.21.15.167:0/1974290650 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610650ceb60 con 0x561065528c00 2024-09-13T23:43:29.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.411+0000 7fee4388a700 1 -- 172.21.15.167:0/1974290650 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5610651b7b80 con 0x561065528c00 2024-09-13T23:43:29.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.411+0000 7fee4388a700 1 --2- 172.21.15.167:0/1974290650 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5610655be000 0x5610655c4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.411+0000 7fee4388a700 1 -- 172.21.15.167:0/1974290650 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x561065123500 con 0x561065528c00 2024-09-13T23:43:29.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.411+0000 7fee60107f40 5 note: GC not initialized 2024-09-13T23:43:29.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.411+0000 7fee5f0c1700 1 --2- 172.21.15.167:0/1974290650 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5610655be000 0x5610655c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.411+0000 7fee5f0c1700 1 --2- 172.21.15.167:0/1974290650 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5610655be000 0x5610655c4000 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x5610654b3590 tx=0x561065425500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.411+0000 7fee41085700 20 reqs_thread_entry: start 2024-09-13T23:43:29.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.411+0000 7fee60107f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:29.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.415+0000 7fee60107f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:29.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.415+0000 7fee3f081700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:59 2024 2024-09-13T23:43:29.423 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:29.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.415+0000 7fee3e85f700 20 reqs_thread_entry: start 2024-09-13T23:43:29.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.415+0000 7fee60107f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:29.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.415+0000 7fee60107f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.415+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065528000 con 0x561065463000 2024-09-13T23:43:29.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.415+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~513 out=513b,getxattrs out=75b] v0'0 uv10 ondisk = 0) v8 ==== 231+0+636 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.415+0000 7fee60107f40 20 rados_obj.operate() r=0 bl.length=513 2024-09-13T23:43:29.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.415+0000 7fee60107f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:29.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.415+0000 7fee60107f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:29.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.415+0000 7fee60107f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:29.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.415+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065522800 con 0x561065463000 2024-09-13T23:43:29.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~357] v33'11 uv11 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee60107f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:29.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee60107f40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:29.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee60107f40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:29.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94628419397888 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523000 con 0x561065463400 2024-09-13T23:43:29.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94628417744128 notify 141733920783 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x561065414680 con 0x561065463400 2024-09-13T23:43:29.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee590b5700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920783 cookie 94628417744128 notifier 4438 bl.length()=533 2024-09-13T23:43:29.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(64 notify.4 [notify cookie 94628419397888 out=8b] v0'0 uv78 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee590b5700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:29.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee590b5700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:29.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee590b5700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523c00 con 0x561065463400 2024-09-13T23:43:29.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94628419397888 notify 141733920783 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x561065414820 con 0x561065463400 2024-09-13T23:43:29.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.419+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 8.2 8:45671577:users.swift::foo%3afoo2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463800 con 0x561065522400 2024-09-13T23:43:29.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.423+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(66 foo:foo2 [delete] v33'2 uv2 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.423+0000 7fee60107f40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-09-13T23:43:29.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.423+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94628419397888 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5610655be400 con 0x561065463400 2024-09-13T23:43:29.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.423+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== watch-notify(notify (1) cookie 94628417739520 notify 141733920784 ret 0) v3 ==== 201+0+0 (crc 0 0 0) 0x561065414820 con 0x561065463400 2024-09-13T23:43:29.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.423+0000 7fee590b5700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920784 cookie 94628417739520 notifier 4438 bl.length()=159 2024-09-13T23:43:29.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.423+0000 7fee590b5700 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065463c00 con 0x561065463400 2024-09-13T23:43:29.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.423+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(67 notify.0 [notify cookie 94628419397888 out=8b] v0'0 uv78 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.423+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.423+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== watch-notify(notify_complete (2) cookie 94628419397888 notify 141733920784 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5610654149c0 con 0x561065463400 2024-09-13T23:43:29.433 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:29.433 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:29.433 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:29.433 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:29.433 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:29.433 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:29.433 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:29.433 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:29.433 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:29.433 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:29.433 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:29.433 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:29.434 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:29.435 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:29.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.423+0000 7fee4588e700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:29.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.423+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94628419395584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5610655be800 con 0x561065463000 2024-09-13T23:43:29.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.427+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94628419395584] v33'87 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.427+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94628417739520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5610655bec00 con 0x561065463400 2024-09-13T23:43:29.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.431+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94628417739520] v33'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.431+0000 7fee60107f40 20 remove_watcher() i=0 2024-09-13T23:43:29.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.431+0000 7fee60107f40 2 removed watcher, disabling cache 2024-09-13T23:43:29.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.431+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94628417740672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5610655bf000 con 0x561065463000 2024-09-13T23:43:29.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.431+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94628417740672] v33'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065463000 2024-09-13T23:43:29.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.431+0000 7fee60107f40 20 remove_watcher() i=1 2024-09-13T23:43:29.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.431+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94628417741824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5610655bf400 con 0x561065463400 2024-09-13T23:43:29.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.435+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94628417741824] v33'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.435+0000 7fee60107f40 20 remove_watcher() i=2 2024-09-13T23:43:29.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.435+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94628417742976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5610655bf800 con 0x561065522400 2024-09-13T23:43:29.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.439+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94628417742976] v33'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.439+0000 7fee60107f40 20 remove_watcher() i=3 2024-09-13T23:43:29.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.439+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94628417744128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5610655bfc00 con 0x561065463400 2024-09-13T23:43:29.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.439+0000 7fee5e8c0700 1 -- 172.21.15.167:0/178873202 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94628417744128] v33'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511200 con 0x561065463400 2024-09-13T23:43:29.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.439+0000 7fee60107f40 20 remove_watcher() i=4 2024-09-13T23:43:29.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.439+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94628417745280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523800 con 0x561065522000 2024-09-13T23:43:29.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.443+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94628417745280] v33'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.443+0000 7fee60107f40 20 remove_watcher() i=5 2024-09-13T23:43:29.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.443+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94628417746432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065523400 con 0x561065522000 2024-09-13T23:43:29.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.443+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94628417746432] v33'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065511440 con 0x561065522000 2024-09-13T23:43:29.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.443+0000 7fee60107f40 20 remove_watcher() i=6 2024-09-13T23:43:29.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.443+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94628417747584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x561065522c00 con 0x561065522400 2024-09-13T23:43:29.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee5f0c1700 1 -- 172.21.15.167:0/178873202 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94628417747584] v33'80 uv78 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x561065510b40 con 0x561065522400 2024-09-13T23:43:29.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 20 remove_watcher() i=7 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561065463400 msgr2=0x5610653a4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x561065463400 0x5610653a4100 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561065463000 msgr2=0x5610653a3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x561065463000 0x5610653a3b80 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561065522400 msgr2=0x5610653a4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x561065522400 0x5610653a4c00 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561065522000 msgr2=0x5610653a4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x561065522000 0x5610653a4680 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561065462c00 msgr2=0x5610653a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561065462c00 0x5610653a3600 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x56106543dd10 tx=0x5610654fa0c0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 msgr2=0x5610653a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 0x5610653a2b00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x5610653e8270 tx=0x5610654245a0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee5e0bf700 1 -- 172.21.15.167:0/178873202 reap_dead start 2024-09-13T23:43:29.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 shutdown_connections 2024-09-13T23:43:29.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 --2- 172.21.15.167:0/178873202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd800 0x5610653a2b00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 >> 172.21.15.167:0/178873202 conn(0x5610650bd400 msgr2=0x5610651bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 shutdown_connections 2024-09-13T23:43:29.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.447+0000 7fee60107f40 1 -- 172.21.15.167:0/178873202 wait complete. 2024-09-13T23:43:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 -- 172.21.15.167:0/1974290650 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5610655be000 msgr2=0x5610655c4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 --2- 172.21.15.167:0/1974290650 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5610655be000 0x5610655c4000 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x5610654b3590 tx=0x561065425500 comp rx=0 tx=0).stop 2024-09-13T23:43:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 -- 172.21.15.167:0/1974290650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 msgr2=0x5610653a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 --2- 172.21.15.167:0/1974290650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 0x5610653a5180 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x561065537350 tx=0x5610654fa180 comp rx=0 tx=0).stop 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 -- 172.21.15.167:0/1974290650 shutdown_connections 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 --2- 172.21.15.167:0/1974290650 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5610655be000 0x5610655c4000 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 --2- 172.21.15.167:0/1974290650 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x561065528c00 0x5610653a5180 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 -- 172.21.15.167:0/1974290650 >> 172.21.15.167:0/1974290650 conn(0x561065529c00 msgr2=0x5610651be000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 -- 172.21.15.167:0/1974290650 shutdown_connections 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 -- 172.21.15.167:0/1974290650 wait complete. 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 -- 172.21.15.167:0/1472314597 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561065462000 msgr2=0x5610653a3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 --2- 172.21.15.167:0/1472314597 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561065462000 0x5610653a3080 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x5610653e94d0 tx=0x561065424c60 comp rx=0 tx=0).stop 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 -- 172.21.15.167:0/1472314597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 msgr2=0x5610653a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 --2- 172.21.15.167:0/1472314597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x561065219f80 tx=0x5610651c1920 comp rx=0 tx=0).stop 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 -- 172.21.15.167:0/1472314597 shutdown_connections 2024-09-13T23:43:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 --2- 172.21.15.167:0/1472314597 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x561065462000 0x5610653a3080 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 --2- 172.21.15.167:0/1472314597 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5610650bd000 0x5610653a2580 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 -- 172.21.15.167:0/1472314597 >> 172.21.15.167:0/1472314597 conn(0x5610650bcc00 msgr2=0x5610651bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 -- 172.21.15.167:0/1472314597 shutdown_connections 2024-09-13T23:43:29.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.451+0000 7fee60107f40 1 -- 172.21.15.167:0/1472314597 wait complete. 2024-09-13T23:43:29.465 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:29.466 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-09-13T23:43:29.466 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-09-13T23:43:29.466 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-09-13T23:43:29.507 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:29.507 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:29.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed5ec1f40 1 Processor -- start 2024-09-13T23:43:29.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed5ec1f40 1 -- start start 2024-09-13T23:43:29.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed5ec1f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56526bf11000 0x56526c1f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed5ec1f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56526bfd5500 con 0x56526bf11400 2024-09-13T23:43:29.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed5ec1f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x56526bf9ea00 con 0x56526bf11000 2024-09-13T23:43:29.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed467a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56526bf11000 0x56526c1f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed467a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56526bf11000 0x56526c1f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51570/0 (socket says 172.21.15.167:51570) 2024-09-13T23:43:29.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed467a700 1 -- 172.21.15.167:0/193182169 learned_addr learned my addr 172.21.15.167:0/193182169 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed3e79700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x56526bf11400 0x56526c010800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45376/0 (socket says 172.21.15.167:45376) 2024-09-13T23:43:29.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed467a700 1 -- 172.21.15.167:0/193182169 >> v1:172.21.15.12:6789/0 conn(0x56526bf11400 legacy=0x56526c010800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.509 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed467a700 1 -- 172.21.15.167:0/193182169 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56526bf23d40 con 0x56526bf11000 2024-09-13T23:43:29.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed467a700 1 --2- 172.21.15.167:0/193182169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x56526c1f1a70 tx=0x56526c015920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659493f964ace16a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed3678700 1 -- 172.21.15.167:0/193182169 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526bfb54a0 con 0x56526bf11000 2024-09-13T23:43:29.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed3678700 1 -- 172.21.15.167:0/193182169 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56526bfd5800 con 0x56526bf11000 2024-09-13T23:43:29.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.499+0000 7f1ed3678700 1 -- 172.21.15.167:0/193182169 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526bfb5600 con 0x56526bf11000 2024-09-13T23:43:29.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/193182169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 msgr2=0x56526c1f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/193182169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x56526c1f1a70 tx=0x56526c015920 comp rx=0 tx=0).stop 2024-09-13T23:43:29.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/193182169 shutdown_connections 2024-09-13T23:43:29.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/193182169 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/193182169 >> 172.21.15.167:0/193182169 conn(0x56526bf10c00 msgr2=0x56526c010000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/193182169 shutdown_connections 2024-09-13T23:43:29.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/193182169 wait complete. 2024-09-13T23:43:29.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 Processor -- start 2024-09-13T23:43:29.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 -- start start 2024-09-13T23:43:29.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526bf9e500 con 0x56526bf11000 2024-09-13T23:43:29.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed467a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed467a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51574/0 (socket says 172.21.15.167:51574) 2024-09-13T23:43:29.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed467a700 1 -- 172.21.15.167:0/3255474472 learned_addr learned my addr 172.21.15.167:0/3255474472 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed467a700 1 -- 172.21.15.167:0/3255474472 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56526bf23ba0 con 0x56526bf11000 2024-09-13T23:43:29.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed467a700 1 --2- 172.21.15.167:0/3255474472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x56526c06c1b0 tx=0x56526c015800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23ab86c915b98df server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed2e77700 1 -- 172.21.15.167:0/3255474472 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526bfb5760 con 0x56526bf11000 2024-09-13T23:43:29.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed2e77700 1 -- 172.21.15.167:0/3255474472 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56526bfd5b00 con 0x56526bf11000 2024-09-13T23:43:29.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed2e77700 1 -- 172.21.15.167:0/3255474472 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526bfb54a0 con 0x56526bf11000 2024-09-13T23:43:29.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/3255474472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 msgr2=0x56526c1f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/3255474472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x56526c06c1b0 tx=0x56526c015800 comp rx=0 tx=0).stop 2024-09-13T23:43:29.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/3255474472 shutdown_connections 2024-09-13T23:43:29.514 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/3255474472 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/3255474472 >> 172.21.15.167:0/3255474472 conn(0x56526bf10c00 msgr2=0x56526c011000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/3255474472 shutdown_connections 2024-09-13T23:43:29.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.503+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/3255474472 wait complete. 2024-09-13T23:43:29.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed5ec1f40 1 Processor -- start 2024-09-13T23:43:29.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed5ec1f40 1 -- start start 2024-09-13T23:43:29.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed5ec1f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed5ec1f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526bf9e3c0 con 0x56526bf11000 2024-09-13T23:43:29.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed467a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed467a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51590/0 (socket says 172.21.15.167:51590) 2024-09-13T23:43:29.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed467a700 1 -- 172.21.15.167:0/480135514 learned_addr learned my addr 172.21.15.167:0/480135514 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed467a700 1 -- 172.21.15.167:0/480135514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56526bf23a00 con 0x56526bf11000 2024-09-13T23:43:29.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed467a700 1 --2- 172.21.15.167:0/480135514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x56526c06df80 tx=0x56526c015920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed1674700 1 -- 172.21.15.167:0/480135514 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526bfb5600 con 0x56526bf11000 2024-09-13T23:43:29.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed1674700 1 -- 172.21.15.167:0/480135514 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56526bfd5c80 con 0x56526bf11000 2024-09-13T23:43:29.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/480135514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56526bf23860 con 0x56526bf11000 2024-09-13T23:43:29.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed1674700 1 -- 172.21.15.167:0/480135514 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526bfb58c0 con 0x56526bf11000 2024-09-13T23:43:29.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/480135514 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56526bf236c0 con 0x56526bf11000 2024-09-13T23:43:29.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed5ec1f40 1 Processor -- start 2024-09-13T23:43:29.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed5ec1f40 1 -- start start 2024-09-13T23:43:29.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.507+0000 7f1ed5ec1f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 0x56526c1f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed5ec1f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526bf9e000 con 0x56526bf11800 2024-09-13T23:43:29.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed1674700 1 -- 172.21.15.167:0/480135514 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56526c00ab00 con 0x56526bf11000 2024-09-13T23:43:29.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed4e7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 0x56526c1f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed1674700 1 --2- 172.21.15.167:0/480135514 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c2b4000 0x56526c1f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed4e7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 0x56526c1f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51598/0 (socket says 172.21.15.167:51598) 2024-09-13T23:43:29.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/4289904751 learned_addr learned my addr 172.21.15.167:0/4289904751 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed1674700 1 -- 172.21.15.167:0/480135514 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56526bf77180 con 0x56526bf11000 2024-09-13T23:43:29.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed467a700 1 --2- 172.21.15.167:0/480135514 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c2b4000 0x56526c1f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.522 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/4289904751 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56526bf22000 con 0x56526bf11800 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed4e7b700 1 --2- 172.21.15.167:0/4289904751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 0x56526c1f6b00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x56526c1f1fb0 tx=0x56526c274b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81d034f9118741e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed467a700 1 --2- 172.21.15.167:0/480135514 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c2b4000 0x56526c1f7080 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x56526c2394d0 tx=0x56526c274c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ecee6f700 1 -- 172.21.15.167:0/4289904751 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526bfb5600 con 0x56526bf11800 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ecee6f700 1 -- 172.21.15.167:0/4289904751 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56526c2d0000 con 0x56526bf11800 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ecee6f700 1 -- 172.21.15.167:0/4289904751 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526bfb58c0 con 0x56526bf11800 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/4289904751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 msgr2=0x56526c1f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/4289904751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 0x56526c1f6b00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x56526c1f1fb0 tx=0x56526c274b40 comp rx=0 tx=0).stop 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/4289904751 shutdown_connections 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/4289904751 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 0x56526c1f6b00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/4289904751 >> 172.21.15.167:0/4289904751 conn(0x56526bf11400 msgr2=0x56526c011800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/4289904751 shutdown_connections 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/4289904751 wait complete. 2024-09-13T23:43:29.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed5ec1f40 1 Processor -- start 2024-09-13T23:43:29.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed5ec1f40 1 -- start start 2024-09-13T23:43:29.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed5ec1f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 0x56526c1f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed5ec1f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526bf36640 con 0x56526bf11800 2024-09-13T23:43:29.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed4e7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 0x56526c1f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed4e7b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 0x56526c1f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51610/0 (socket says 172.21.15.167:51610) 2024-09-13T23:43:29.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.511+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 learned_addr learned my addr 172.21.15.167:0/544817134 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56526bf22ea0 con 0x56526bf11800 2024-09-13T23:43:29.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ed4e7b700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 0x56526c1f6b00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x56526c238840 tx=0x56526c2745a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ecd66c700 1 -- 172.21.15.167:0/544817134 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526bfb5760 con 0x56526bf11800 2024-09-13T23:43:29.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ecd66c700 1 -- 172.21.15.167:0/544817134 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56526c2d0180 con 0x56526bf11800 2024-09-13T23:43:29.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56526bf22820 con 0x56526bf11800 2024-09-13T23:43:29.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ecd66c700 1 -- 172.21.15.167:0/544817134 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526bfb5600 con 0x56526bf11800 2024-09-13T23:43:29.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56526bf229c0 con 0x56526bf11800 2024-09-13T23:43:29.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ed5ec1f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ecd66c700 1 -- 172.21.15.167:0/544817134 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56526c00b080 con 0x56526bf11800 2024-09-13T23:43:29.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ecd66c700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c2b4c00 0x56526c1f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ecd66c700 1 -- 172.21.15.167:0/544817134 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56526bf77340 con 0x56526bf11800 2024-09-13T23:43:29.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ed3e79700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c2b4c00 0x56526c1f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56526c2b5000 0x56526c1f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5400 con 0x56526c2b5000 2024-09-13T23:43:29.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.515+0000 7f1ed4e7b700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56526c2b5000 0x56526c1f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed3e79700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c2b4c00 0x56526c1f7600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x56526c1f1f80 tx=0x56526c36a0c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed4e7b700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56526c2b5000 0x56526c1f7b80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed5ec1f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed5ec1f40 20 realm 2024-09-13T23:43:29.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed5ec1f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5000 2024-09-13T23:43:29.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed5ec1f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed5ec1f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:29.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed5ec1f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5c00 con 0x56526c2b5000 2024-09-13T23:43:29.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed5ec1f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed5ec1f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56526c2b5400 0x56526c1f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5400 2024-09-13T23:43:29.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed467a700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56526c2b5400 0x56526c1f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.519+0000 7f1ed467a700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56526c2b5400 0x56526c1f8100 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed5ec1f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:29.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed5ec1f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5c00 con 0x56526c2b5400 2024-09-13T23:43:29.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed5ec1f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:29.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed5ec1f40 20 searching for the correct realm 2024-09-13T23:43:29.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56526c24c000 0x56526c1f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24c400 con 0x56526c24c000 2024-09-13T23:43:29.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed3e79700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56526c24c000 0x56526c1f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed3e79700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56526c24c000 0x56526c1f8680 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24c400 con 0x56526c2b5400 2024-09-13T23:43:29.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed467a700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56526c24c400 0x56526c1f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c400 2024-09-13T23:43:29.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.523+0000 7f1ed4e7b700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56526c24c400 0x56526c1f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.527+0000 7f1ed4e7b700 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56526c24c400 0x56526c1f8c00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.527+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.527+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5000 2024-09-13T23:43:29.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.527+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.527+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5000 2024-09-13T23:43:29.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.527+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.527+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c000 2024-09-13T23:43:29.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.527+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.527+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5000 2024-09-13T23:43:29.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.527+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.527+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5400 2024-09-13T23:43:29.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.531+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.531+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5400 2024-09-13T23:43:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.531+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.531+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5000 2024-09-13T23:43:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.531+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.531+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c400 2024-09-13T23:43:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.531+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.531+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5000 2024-09-13T23:43:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.531+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.531+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c400 2024-09-13T23:43:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.531+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.535+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5400 2024-09-13T23:43:29.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.535+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.535+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c000 2024-09-13T23:43:29.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.535+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.535+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5000 2024-09-13T23:43:29.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.535+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.535+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5400 2024-09-13T23:43:29.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.535+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.535+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c000 2024-09-13T23:43:29.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.535+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.535+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5000 2024-09-13T23:43:29.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.535+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5000 2024-09-13T23:43:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5000 2024-09-13T23:43:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c000 2024-09-13T23:43:29.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c400 2024-09-13T23:43:29.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5000 2024-09-13T23:43:29.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c400 2024-09-13T23:43:29.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.539+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c000 2024-09-13T23:43:29.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.543+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.543+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c400 2024-09-13T23:43:29.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.543+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.543+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5400 2024-09-13T23:43:29.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.543+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.543+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5400 2024-09-13T23:43:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.543+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.543+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c000 2024-09-13T23:43:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.543+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.543+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5400 2024-09-13T23:43:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5400 2024-09-13T23:43:29.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:29.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:29.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:29.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:29.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24c800 con 0x56526c2b5000 2024-09-13T23:43:29.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24cc00 con 0x56526c2b5400 2024-09-13T23:43:29.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:29.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24d000 con 0x56526c2b5400 2024-09-13T23:43:29.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:29.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 zone default found 2024-09-13T23:43:29.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 4 Realm: () 2024-09-13T23:43:29.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:29.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:29.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:29.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 zonegroup default 2024-09-13T23:43:29.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24d400 con 0x56526c24c400 2024-09-13T23:43:29.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.547+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24d800 con 0x56526c24c000 2024-09-13T23:43:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.551+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.551+0000 7f1ed5ec1f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.551+0000 7f1ed5ec1f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.551+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24dc00 con 0x56526c24c000 2024-09-13T23:43:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.551+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.551+0000 7f1ed5ec1f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.551+0000 7f1ed5ec1f40 20 started sync module instance, tier type = 2024-09-13T23:43:29.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.551+0000 7f1ed5ec1f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:29.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.551+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24d800 con 0x56526c2b5400 2024-09-13T23:43:29.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.555+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.555+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94912001268480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24dc00 con 0x56526c2b5400 2024-09-13T23:43:29.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.555+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24cc00 con 0x56526c2b5000 2024-09-13T23:43:29.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.555+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94912001268480] v33'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.555+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.555+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94912001269632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24d000 con 0x56526c2b5000 2024-09-13T23:43:29.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.555+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24d800 con 0x56526c2b5400 2024-09-13T23:43:29.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.559+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94912001269632] v33'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.559+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.559+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94912001270784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5c00 con 0x56526c2b5400 2024-09-13T23:43:29.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.559+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c24c400 2024-09-13T23:43:29.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.563+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.563+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94912001271936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24c800 con 0x56526c24c400 2024-09-13T23:43:29.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.563+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24d400 con 0x56526c2b5400 2024-09-13T23:43:29.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.563+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94912001270784] v33'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.563+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94912001271936] v33'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.563+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.563+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94912001273088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24cc00 con 0x56526c2b5400 2024-09-13T23:43:29.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.563+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24d000 con 0x56526c24c000 2024-09-13T23:43:29.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.567+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94912001273088] v33'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.567+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.567+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94912001274240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24e000 con 0x56526c24c000 2024-09-13T23:43:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.567+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24e400 con 0x56526c24c000 2024-09-13T23:43:29.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94912001274240] v33'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94912001275392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24e800 con 0x56526c24c000 2024-09-13T23:43:29.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24ec00 con 0x56526c24c400 2024-09-13T23:43:29.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'81 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94912001276544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24f000 con 0x56526c24c400 2024-09-13T23:43:29.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed5ec1f40 20 add_watcher() i=0 2024-09-13T23:43:29.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed5ec1f40 20 add_watcher() i=1 2024-09-13T23:43:29.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed5ec1f40 20 add_watcher() i=2 2024-09-13T23:43:29.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed5ec1f40 20 add_watcher() i=3 2024-09-13T23:43:29.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed5ec1f40 20 add_watcher() i=4 2024-09-13T23:43:29.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed5ec1f40 20 add_watcher() i=5 2024-09-13T23:43:29.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94912001275392] v33'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.571+0000 7f1ed5ec1f40 20 add_watcher() i=6 2024-09-13T23:43:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.575+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94912001276544] v33'82 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.575+0000 7f1ed5ec1f40 20 add_watcher() i=7 2024-09-13T23:43:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.575+0000 7f1ed5ec1f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:29.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.575+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24f400 con 0x56526c2b5000 2024-09-13T23:43:29.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.575+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.575+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94912002957312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24f800 con 0x56526c2b5000 2024-09-13T23:43:29.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.575+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94912002957312] v33'88 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.575+0000 7f1ebb648700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:29.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.575+0000 7f1ed5ec1f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:29.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.575+0000 7f1ed5ec1f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:29.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.575+0000 7f1ed5ec1f40 1 Processor -- start 2024-09-13T23:43:29.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed5ec1f40 1 -- start start 2024-09-13T23:43:29.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed5ec1f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 0x56526c1f9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed5ec1f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526bf36780 con 0x56526c24ec00 2024-09-13T23:43:29.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed3e79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 0x56526c1f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed3e79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 0x56526c1f9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51618/0 (socket says 172.21.15.167:51618) 2024-09-13T23:43:29.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed3e79700 1 -- 172.21.15.167:0/634405297 learned_addr learned my addr 172.21.15.167:0/634405297 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed3e79700 1 -- 172.21.15.167:0/634405297 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56526bf22b60 con 0x56526c24ec00 2024-09-13T23:43:29.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed3e79700 1 --2- 172.21.15.167:0/634405297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 0x56526c1f9180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x56526c394060 tx=0x56526c36a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bec33aa90c0559 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ebae47700 1 -- 172.21.15.167:0/634405297 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526c39c000 con 0x56526c24ec00 2024-09-13T23:43:29.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ebae47700 1 -- 172.21.15.167:0/634405297 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56526c2d0600 con 0x56526c24ec00 2024-09-13T23:43:29.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ebae47700 1 -- 172.21.15.167:0/634405297 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526c39c160 con 0x56526c24ec00 2024-09-13T23:43:29.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/634405297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 msgr2=0x56526c1f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/634405297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 0x56526c1f9180 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x56526c394060 tx=0x56526c36a060 comp rx=0 tx=0).stop 2024-09-13T23:43:29.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/634405297 shutdown_connections 2024-09-13T23:43:29.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/634405297 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 0x56526c1f9180 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/634405297 >> 172.21.15.167:0/634405297 conn(0x56526c24fc00 msgr2=0x56526c012000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/634405297 shutdown_connections 2024-09-13T23:43:29.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/634405297 wait complete. 2024-09-13T23:43:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.579+0000 7f1ed5ec1f40 1 Processor -- start 2024-09-13T23:43:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1ed5ec1f40 1 -- start start 2024-09-13T23:43:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1ed5ec1f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 0x56526c1f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1ed5ec1f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56526bf9e000 con 0x56526c24ec00 2024-09-13T23:43:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1ed3e79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 0x56526c1f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1ed3e79700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 0x56526c1f9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51628/0 (socket says 172.21.15.167:51628) 2024-09-13T23:43:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1ed3e79700 1 -- 172.21.15.167:0/2713230137 learned_addr learned my addr 172.21.15.167:0/2713230137 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1ed3e79700 1 -- 172.21.15.167:0/2713230137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56526bf22d00 con 0x56526c24ec00 2024-09-13T23:43:29.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1ed3e79700 1 --2- 172.21.15.167:0/2713230137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 0x56526c1f9180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x56526c394d20 tx=0x56526c36a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1eb9644700 1 -- 172.21.15.167:0/2713230137 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526c39c2c0 con 0x56526c24ec00 2024-09-13T23:43:29.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1eb9644700 1 -- 172.21.15.167:0/2713230137 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56526c2d0300 con 0x56526c24ec00 2024-09-13T23:43:29.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/2713230137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56526bf22340 con 0x56526c24ec00 2024-09-13T23:43:29.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1eb9644700 1 -- 172.21.15.167:0/2713230137 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56526c39c000 con 0x56526c24ec00 2024-09-13T23:43:29.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/2713230137 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56526bf22b60 con 0x56526c24ec00 2024-09-13T23:43:29.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.583+0000 7f1eb9644700 1 -- 172.21.15.167:0/2713230137 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56526c00bb80 con 0x56526c24ec00 2024-09-13T23:43:29.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1eb9644700 1 --2- 172.21.15.167:0/2713230137 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c416000 0x56526c41c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1eb9644700 1 -- 172.21.15.167:0/2713230137 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56526bf77500 con 0x56526c24ec00 2024-09-13T23:43:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1ed5ec1f40 5 note: GC not initialized 2024-09-13T23:43:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1ed4e7b700 1 --2- 172.21.15.167:0/2713230137 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c416000 0x56526c41c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1ed4e7b700 1 --2- 172.21.15.167:0/2713230137 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c416000 0x56526c41c000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x56526c32b020 tx=0x56526c275500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1eb6e3f700 20 reqs_thread_entry: start 2024-09-13T23:43:29.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1ed5ec1f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:29.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1ed5ec1f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:29.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1eb4e3b700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:59 2024 2024-09-13T23:43:29.597 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1eb4619700 20 reqs_thread_entry: start 2024-09-13T23:43:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1ed5ec1f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24e000 con 0x56526c2b5000 2024-09-13T23:43:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.587+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.591+0000 7f1ed5ec1f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.591+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24e400 con 0x56526c24c400 2024-09-13T23:43:29.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.591+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.591+0000 7f1ebb648700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:29.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.591+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94912002957312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24c800 con 0x56526c2b5000 2024-09-13T23:43:29.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.595+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94912002957312] v33'89 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.595+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94912001268480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c24d000 con 0x56526c2b5400 2024-09-13T23:43:29.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.595+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94912001268480] v33'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.595+0000 7f1ed5ec1f40 20 remove_watcher() i=0 2024-09-13T23:43:29.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.595+0000 7f1ed5ec1f40 2 removed watcher, disabling cache 2024-09-13T23:43:29.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.595+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94912001269632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c2b5800 con 0x56526c2b5000 2024-09-13T23:43:29.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.599+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94912001269632] v33'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c2b5000 2024-09-13T23:43:29.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.599+0000 7f1ed5ec1f40 20 remove_watcher() i=1 2024-09-13T23:43:29.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.599+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94912001270784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c416400 con 0x56526c2b5400 2024-09-13T23:43:29.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.599+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94912001270784] v33'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.599+0000 7f1ed5ec1f40 20 remove_watcher() i=2 2024-09-13T23:43:29.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.599+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94912001271936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c416800 con 0x56526c24c400 2024-09-13T23:43:29.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.603+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94912001271936] v33'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.603+0000 7f1ed5ec1f40 20 remove_watcher() i=3 2024-09-13T23:43:29.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.603+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94912001273088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c416c00 con 0x56526c2b5400 2024-09-13T23:43:29.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.603+0000 7f1ed467a700 1 -- 172.21.15.167:0/544817134 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94912001273088] v33'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321200 con 0x56526c2b5400 2024-09-13T23:43:29.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.603+0000 7f1ed5ec1f40 20 remove_watcher() i=4 2024-09-13T23:43:29.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.603+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94912001274240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c417000 con 0x56526c24c000 2024-09-13T23:43:29.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.607+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94912001274240] v33'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.607+0000 7f1ed5ec1f40 20 remove_watcher() i=5 2024-09-13T23:43:29.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.607+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94912001275392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c417400 con 0x56526c24c000 2024-09-13T23:43:29.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94912001275392] v33'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c321440 con 0x56526c24c000 2024-09-13T23:43:29.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 20 remove_watcher() i=6 2024-09-13T23:43:29.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94912001276544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x56526c417800 con 0x56526c24c400 2024-09-13T23:43:29.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed4e7b700 1 -- 172.21.15.167:0/544817134 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94912001276544] v33'83 uv81 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56526c320b40 con 0x56526c24c400 2024-09-13T23:43:29.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 20 remove_watcher() i=7 2024-09-13T23:43:29.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56526c2b5400 msgr2=0x56526c1f8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56526c2b5400 0x56526c1f8100 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56526c2b5000 msgr2=0x56526c1f7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56526c2b5000 0x56526c1f7b80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56526c24c400 msgr2=0x56526c1f8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56526c24c400 0x56526c1f8c00 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56526c24c000 msgr2=0x56526c1f8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56526c24c000 0x56526c1f8680 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c2b4c00 msgr2=0x56526c1f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c2b4c00 0x56526c1f7600 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x56526c1f1f80 tx=0x56526c36a0c0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 msgr2=0x56526c1f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/544817134 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11800 0x56526c1f6b00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x56526c238840 tx=0x56526c2745a0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 reap_dead start 2024-09-13T23:43:29.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed3e79700 1 -- 172.21.15.167:0/544817134 reap_dead start 2024-09-13T23:43:29.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 shutdown_connections 2024-09-13T23:43:29.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 >> 172.21.15.167:0/544817134 conn(0x56526bf11400 msgr2=0x56526c011800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 shutdown_connections 2024-09-13T23:43:29.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.611+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/544817134 wait complete. 2024-09-13T23:43:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/2713230137 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c416000 msgr2=0x56526c41c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/2713230137 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c416000 0x56526c41c000 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x56526c32b020 tx=0x56526c275500 comp rx=0 tx=0).stop 2024-09-13T23:43:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/2713230137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 msgr2=0x56526c1f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/2713230137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 0x56526c1f9180 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x56526c394d20 tx=0x56526c36a180 comp rx=0 tx=0).stop 2024-09-13T23:43:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/2713230137 shutdown_connections 2024-09-13T23:43:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/2713230137 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c416000 0x56526c41c000 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/2713230137 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526c24ec00 0x56526c1f9180 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/2713230137 >> 172.21.15.167:0/2713230137 conn(0x56526c24fc00 msgr2=0x56526c012000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/2713230137 shutdown_connections 2024-09-13T23:43:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/2713230137 wait complete. 2024-09-13T23:43:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/480135514 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c2b4000 msgr2=0x56526c1f7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/480135514 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c2b4000 0x56526c1f7080 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x56526c2394d0 tx=0x56526c274c60 comp rx=0 tx=0).stop 2024-09-13T23:43:29.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/480135514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 msgr2=0x56526c1f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/480135514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x56526c06df80 tx=0x56526c015920 comp rx=0 tx=0).stop 2024-09-13T23:43:29.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/480135514 shutdown_connections 2024-09-13T23:43:29.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/480135514 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56526c2b4000 0x56526c1f7080 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 --2- 172.21.15.167:0/480135514 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56526bf11000 0x56526c1f6580 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/480135514 >> 172.21.15.167:0/480135514 conn(0x56526bf10c00 msgr2=0x56526c011000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/480135514 shutdown_connections 2024-09-13T23:43:29.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.615+0000 7f1ed5ec1f40 1 -- 172.21.15.167:0/480135514 wait complete. 2024-09-13T23:43:29.626 INFO:teuthology.orchestra.run.smithi167.stdout:[] 2024-09-13T23:43:29.629 DEBUG:tasks.util.rgw: json result: [] 2024-09-13T23:43:29.629 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-09-13T23:43:29.629 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-09-13T23:43:29.629 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-09-13T23:43:29.717 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:29.717 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:29.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.707+0000 7f5a30fbdf40 1 Processor -- start 2024-09-13T23:43:29.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.707+0000 7f5a30fbdf40 1 -- start start 2024-09-13T23:43:29.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.707+0000 7f5a30fbdf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55e2416f9400 0x55e2419de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.707+0000 7f5a30fbdf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e2417bd500 con 0x55e2416f9000 2024-09-13T23:43:29.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.707+0000 7f5a30fbdf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e241786a00 con 0x55e2416f9400 2024-09-13T23:43:29.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a2ef75700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55e2416f9400 0x55e2419de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a2f776700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55e2416f9000 0x55e2417f8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45386/0 (socket says 172.21.15.167:45386) 2024-09-13T23:43:29.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a2f776700 1 -- 172.21.15.167:0/2654951724 learned_addr learned my addr 172.21.15.167:0/2654951724 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:29.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a2ef75700 1 -- 172.21.15.167:0/2654951724 >> v1:172.21.15.12:6789/0 conn(0x55e2416f9000 legacy=0x55e2417f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a2ef75700 1 -- 172.21.15.167:0/2654951724 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24170bd40 con 0x55e2416f9400 2024-09-13T23:43:29.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a2ef75700 1 --2- 172.21.15.167:0/2654951724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9400 0x55e2419de580 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55e2419d9920 tx=0x55e2417fd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48606ce2c398b1cf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a2e774700 1 -- 172.21.15.167:0/2654951724 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e24179d4a0 con 0x55e2416f9400 2024-09-13T23:43:29.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a2e774700 1 -- 172.21.15.167:0/2654951724 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e2417bd800 con 0x55e2416f9400 2024-09-13T23:43:29.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a2e774700 1 -- 172.21.15.167:0/2654951724 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e24179d600 con 0x55e2416f9400 2024-09-13T23:43:29.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/2654951724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9400 msgr2=0x55e2419de580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/2654951724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9400 0x55e2419de580 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55e2419d9920 tx=0x55e2417fd920 comp rx=0 tx=0).stop 2024-09-13T23:43:29.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/2654951724 shutdown_connections 2024-09-13T23:43:29.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/2654951724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9400 0x55e2419de580 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/2654951724 >> 172.21.15.167:0/2654951724 conn(0x55e2416f8c00 msgr2=0x55e2417f8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/2654951724 shutdown_connections 2024-09-13T23:43:29.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/2654951724 wait complete. 2024-09-13T23:43:29.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a30fbdf40 1 Processor -- start 2024-09-13T23:43:29.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a30fbdf40 1 -- start start 2024-09-13T23:43:29.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a30fbdf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 0x55e2419de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a30fbdf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e241786500 con 0x55e2416f9000 2024-09-13T23:43:29.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a2f776700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 0x55e2419de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a2f776700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 0x55e2419de580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51640/0 (socket says 172.21.15.167:51640) 2024-09-13T23:43:29.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.711+0000 7f5a2f776700 1 -- 172.21.15.167:0/578157820 learned_addr learned my addr 172.21.15.167:0/578157820 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a2f776700 1 -- 172.21.15.167:0/578157820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24170bba0 con 0x55e2416f9000 2024-09-13T23:43:29.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a2f776700 1 --2- 172.21.15.167:0/578157820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 0x55e2419de580 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55e2424bbe60 tx=0x55e2417fdb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5d608eb96a39e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a2df73700 1 -- 172.21.15.167:0/578157820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e24179d4a0 con 0x55e2416f9000 2024-09-13T23:43:29.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a2df73700 1 -- 172.21.15.167:0/578157820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e2417bdb00 con 0x55e2416f9000 2024-09-13T23:43:29.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a2df73700 1 -- 172.21.15.167:0/578157820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e24179d600 con 0x55e2416f9000 2024-09-13T23:43:29.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/578157820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 msgr2=0x55e2419de580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/578157820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 0x55e2419de580 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x55e2424bbe60 tx=0x55e2417fdb60 comp rx=0 tx=0).stop 2024-09-13T23:43:29.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/578157820 shutdown_connections 2024-09-13T23:43:29.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/578157820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 0x55e2419de580 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/578157820 >> 172.21.15.167:0/578157820 conn(0x55e2416f8c00 msgr2=0x55e2417f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/578157820 shutdown_connections 2024-09-13T23:43:29.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/578157820 wait complete. 2024-09-13T23:43:29.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a30fbdf40 1 Processor -- start 2024-09-13T23:43:29.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a30fbdf40 1 -- start start 2024-09-13T23:43:29.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a30fbdf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 0x55e2419de580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a30fbdf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e2417863c0 con 0x55e2416f9000 2024-09-13T23:43:29.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a2f776700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 0x55e2419de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a2f776700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 0x55e2419de580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51642/0 (socket says 172.21.15.167:51642) 2024-09-13T23:43:29.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a2f776700 1 -- 172.21.15.167:0/1339317086 learned_addr learned my addr 172.21.15.167:0/1339317086 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a2f776700 1 -- 172.21.15.167:0/1339317086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24170ba00 con 0x55e2416f9000 2024-09-13T23:43:29.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.715+0000 7f5a2f776700 1 --2- 172.21.15.167:0/1339317086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 0x55e2419de580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55e2424bbec0 tx=0x55e2417fda40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2c770700 1 -- 172.21.15.167:0/1339317086 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e24179d8c0 con 0x55e2416f9000 2024-09-13T23:43:29.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2c770700 1 -- 172.21.15.167:0/1339317086 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e2417bdc80 con 0x55e2416f9000 2024-09-13T23:43:29.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/1339317086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e24170b860 con 0x55e2416f9000 2024-09-13T23:43:29.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2c770700 1 -- 172.21.15.167:0/1339317086 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e24179d4a0 con 0x55e2416f9000 2024-09-13T23:43:29.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/1339317086 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e24170b6c0 con 0x55e2416f9000 2024-09-13T23:43:29.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2c770700 1 -- 172.21.15.167:0/1339317086 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e2417f2b00 con 0x55e2416f9000 2024-09-13T23:43:29.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a30fbdf40 1 Processor -- start 2024-09-13T23:43:29.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a30fbdf40 1 -- start start 2024-09-13T23:43:29.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2c770700 1 --2- 172.21.15.167:0/1339317086 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241a98000 0x55e2419df080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2c770700 1 -- 172.21.15.167:0/1339317086 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e24175f180 con 0x55e2416f9000 2024-09-13T23:43:29.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a30fbdf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 0x55e2419deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a30fbdf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e241786000 con 0x55e2416f9800 2024-09-13T23:43:29.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2ff77700 1 --2- 172.21.15.167:0/1339317086 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241a98000 0x55e2419df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2f776700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 0x55e2419deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2f776700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 0x55e2419deb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51650/0 (socket says 172.21.15.167:51650) 2024-09-13T23:43:29.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2f776700 1 -- 172.21.15.167:0/4196748259 learned_addr learned my addr 172.21.15.167:0/4196748259 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2ff77700 1 --2- 172.21.15.167:0/1339317086 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241a98000 0x55e2419df080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55e242570b10 tx=0x55e241a52de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2f776700 1 -- 172.21.15.167:0/4196748259 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24170a000 con 0x55e2416f9800 2024-09-13T23:43:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.719+0000 7f5a2f776700 1 --2- 172.21.15.167:0/4196748259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 0x55e2419deb00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55e2425703c0 tx=0x55e241a52240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300a8fd34aa78237 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a29f6b700 1 -- 172.21.15.167:0/4196748259 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e24179d600 con 0x55e2416f9800 2024-09-13T23:43:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a29f6b700 1 -- 172.21.15.167:0/4196748259 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e2417bdc80 con 0x55e2416f9800 2024-09-13T23:43:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a29f6b700 1 -- 172.21.15.167:0/4196748259 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e24179da20 con 0x55e2416f9800 2024-09-13T23:43:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/4196748259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 msgr2=0x55e2419deb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/4196748259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 0x55e2419deb00 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55e2425703c0 tx=0x55e241a52240 comp rx=0 tx=0).stop 2024-09-13T23:43:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/4196748259 shutdown_connections 2024-09-13T23:43:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/4196748259 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 0x55e2419deb00 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/4196748259 >> 172.21.15.167:0/4196748259 conn(0x55e2416f9400 msgr2=0x55e2417f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/4196748259 shutdown_connections 2024-09-13T23:43:29.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/4196748259 wait complete. 2024-09-13T23:43:29.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 Processor -- start 2024-09-13T23:43:29.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 -- start start 2024-09-13T23:43:29.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 0x55e2419deb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24171e640 con 0x55e2416f9800 2024-09-13T23:43:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a2f776700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 0x55e2419deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a2f776700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 0x55e2419deb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51656/0 (socket says 172.21.15.167:51656) 2024-09-13T23:43:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 learned_addr learned my addr 172.21.15.167:0/3599133295 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24170aea0 con 0x55e2416f9800 2024-09-13T23:43:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a2f776700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 0x55e2419deb00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55e241a9ae40 tx=0x55e241a52fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a28768700 1 -- 172.21.15.167:0/3599133295 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e24179d600 con 0x55e2416f9800 2024-09-13T23:43:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a28768700 1 -- 172.21.15.167:0/3599133295 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e241ac2000 con 0x55e2416f9800 2024-09-13T23:43:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e24170a820 con 0x55e2416f9800 2024-09-13T23:43:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a28768700 1 -- 172.21.15.167:0/3599133295 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e24179da20 con 0x55e2416f9800 2024-09-13T23:43:29.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.723+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e24170a9c0 con 0x55e2416f9800 2024-09-13T23:43:29.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a28768700 1 -- 172.21.15.167:0/3599133295 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e2417f2dc0 con 0x55e2416f9800 2024-09-13T23:43:29.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a28768700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241a99000 0x55e2419df600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a28768700 1 -- 172.21.15.167:0/3599133295 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e24175f340 con 0x55e2416f9800 2024-09-13T23:43:29.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e241a99400 0x55e2419dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241a99800 con 0x55e241a99400 2024-09-13T23:43:29.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a2ef75700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241a99000 0x55e2419df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a2ff77700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e241a99400 0x55e2419dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a2ef75700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241a99000 0x55e2419df600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55e2419d9920 tx=0x55e2417fd800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a2ff77700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e241a99400 0x55e2419dfb80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 20 realm 2024-09-13T23:43:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241a99c00 con 0x55e241a99400 2024-09-13T23:43:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241a99800 con 0x55e241a99400 2024-09-13T23:43:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e241a99c00 0x55e2419e0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e000 con 0x55e241a99c00 2024-09-13T23:43:29.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.727+0000 7f5a2f776700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e241a99c00 0x55e2419e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a2f776700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e241a99c00 0x55e2419e0100 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a30fbdf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:29.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a30fbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99c00 2024-09-13T23:43:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a30fbdf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a30fbdf40 20 searching for the correct realm 2024-09-13T23:43:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e241b2e800 0x55e2419e0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2ec00 con 0x55e241b2e800 2024-09-13T23:43:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a2ef75700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e241b2e800 0x55e2419e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a2ef75700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e241b2e800 0x55e2419e0680 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.731+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2ec00 con 0x55e241a99c00 2024-09-13T23:43:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2f776700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e241b2e000 0x55e2419e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e000 2024-09-13T23:43:29.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2ff77700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e241b2e000 0x55e2419e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2ff77700 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e241b2e000 0x55e2419e0c00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99400 2024-09-13T23:43:29.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99400 2024-09-13T23:43:29.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e800 2024-09-13T23:43:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99400 2024-09-13T23:43:29.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.735+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99c00 2024-09-13T23:43:29.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99c00 2024-09-13T23:43:29.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99400 2024-09-13T23:43:29.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e000 2024-09-13T23:43:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99400 2024-09-13T23:43:29.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e000 2024-09-13T23:43:29.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99c00 2024-09-13T23:43:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.739+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e800 2024-09-13T23:43:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99400 2024-09-13T23:43:29.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99c00 2024-09-13T23:43:29.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e800 2024-09-13T23:43:29.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99400 2024-09-13T23:43:29.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99400 2024-09-13T23:43:29.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99400 2024-09-13T23:43:29.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.743+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e800 2024-09-13T23:43:29.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e000 2024-09-13T23:43:29.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99400 2024-09-13T23:43:29.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e000 2024-09-13T23:43:29.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e800 2024-09-13T23:43:29.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e000 2024-09-13T23:43:29.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99c00 2024-09-13T23:43:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.747+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99c00 2024-09-13T23:43:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e800 2024-09-13T23:43:29.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99c00 2024-09-13T23:43:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99c00 2024-09-13T23:43:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a30fbdf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:29.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a30fbdf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:29.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a30fbdf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:29.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a30fbdf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:29.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a30fbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2f000 con 0x55e241a99400 2024-09-13T23:43:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a30fbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a30fbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.751+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2f400 con 0x55e241a99c00 2024-09-13T23:43:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2f800 con 0x55e241a99c00 2024-09-13T23:43:29.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 zone default found 2024-09-13T23:43:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 4 Realm: () 2024-09-13T23:43:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 zonegroup default 2024-09-13T23:43:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2fc00 con 0x55e241b2e000 2024-09-13T23:43:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2f000 con 0x55e241b2e800 2024-09-13T23:43:29.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2fc00 con 0x55e241b2e800 2024-09-13T23:43:29.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 started sync module instance, tier type = 2024-09-13T23:43:29.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:29.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.755+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241a99800 con 0x55e241a99c00 2024-09-13T23:43:29.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.759+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.759+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94430251801344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2ec00 con 0x55e241a99c00 2024-09-13T23:43:29.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.759+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241a99400 2024-09-13T23:43:29.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.763+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.763+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94430251802496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2f400 con 0x55e241a99400 2024-09-13T23:43:29.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.763+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2f800 con 0x55e241a99c00 2024-09-13T23:43:29.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.763+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94430251801344] v33'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.763+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.763+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94430251802496] v33'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.763+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94430251803648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2f000 con 0x55e241a99c00 2024-09-13T23:43:29.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.763+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2fc00 con 0x55e241b2e000 2024-09-13T23:43:29.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.767+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94430251803648] v33'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.767+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.767+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94430251804800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241a98800 con 0x55e241b2e000 2024-09-13T23:43:29.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.767+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241a99800 con 0x55e241a99c00 2024-09-13T23:43:29.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.771+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94430251804800] v33'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.771+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.771+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94430251805952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241a98800 con 0x55e241a99c00 2024-09-13T23:43:29.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.771+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e800 2024-09-13T23:43:29.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.771+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.771+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94430251807104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2f400 con 0x55e241b2e800 2024-09-13T23:43:29.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.771+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2fc00 con 0x55e241b2e800 2024-09-13T23:43:29.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.775+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94430251805952] v33'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.775+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.775+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94430251808256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2e400 con 0x55e241b2e800 2024-09-13T23:43:29.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.775+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2f400 con 0x55e241b2e000 2024-09-13T23:43:29.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.775+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94430251807104] v33'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.775+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'84 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.775+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94430251809408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b2fc00 con 0x55e241b2e000 2024-09-13T23:43:29.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.775+0000 7f5a30fbdf40 20 add_watcher() i=0 2024-09-13T23:43:29.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.775+0000 7f5a30fbdf40 20 add_watcher() i=1 2024-09-13T23:43:29.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.779+0000 7f5a30fbdf40 20 add_watcher() i=2 2024-09-13T23:43:29.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.779+0000 7f5a30fbdf40 20 add_watcher() i=3 2024-09-13T23:43:29.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.779+0000 7f5a30fbdf40 20 add_watcher() i=4 2024-09-13T23:43:29.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.779+0000 7f5a30fbdf40 20 add_watcher() i=5 2024-09-13T23:43:29.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.779+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94430251808256] v33'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.779+0000 7f5a30fbdf40 20 add_watcher() i=6 2024-09-13T23:43:29.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.779+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94430251809408] v33'85 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.779+0000 7f5a30fbdf40 20 add_watcher() i=7 2024-09-13T23:43:29.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.779+0000 7f5a30fbdf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:29.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.779+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b40000 con 0x55e241a99400 2024-09-13T23:43:29.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.779+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.779+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94430253244416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b40400 con 0x55e241a99400 2024-09-13T23:43:29.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94430253244416] v33'90 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a16744700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:29.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a30fbdf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:29.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a30fbdf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:29.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a30fbdf40 1 Processor -- start 2024-09-13T23:43:29.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a30fbdf40 1 -- start start 2024-09-13T23:43:29.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a30fbdf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 0x55e2419e1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a30fbdf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24171e780 con 0x55e241b40c00 2024-09-13T23:43:29.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a2ef75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 0x55e2419e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a2ef75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 0x55e2419e1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51664/0 (socket says 172.21.15.167:51664) 2024-09-13T23:43:29.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a2ef75700 1 -- 172.21.15.167:0/982934715 learned_addr learned my addr 172.21.15.167:0/982934715 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a2ef75700 1 -- 172.21.15.167:0/982934715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24170ab60 con 0x55e241b40c00 2024-09-13T23:43:29.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a2ef75700 1 --2- 172.21.15.167:0/982934715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 0x55e2419e1180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55e241ad4090 tx=0x55e2417fd860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83439f8c0c3314a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a15f43700 1 -- 172.21.15.167:0/982934715 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e24179d760 con 0x55e241b40c00 2024-09-13T23:43:29.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a15f43700 1 -- 172.21.15.167:0/982934715 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e241ac2900 con 0x55e241b40c00 2024-09-13T23:43:29.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.783+0000 7f5a15f43700 1 -- 172.21.15.167:0/982934715 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e241b42160 con 0x55e241b40c00 2024-09-13T23:43:29.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/982934715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 msgr2=0x55e2419e1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/982934715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 0x55e2419e1180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x55e241ad4090 tx=0x55e2417fd860 comp rx=0 tx=0).stop 2024-09-13T23:43:29.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/982934715 shutdown_connections 2024-09-13T23:43:29.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/982934715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 0x55e2419e1180 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/982934715 >> 172.21.15.167:0/982934715 conn(0x55e241b40800 msgr2=0x55e2417fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/982934715 shutdown_connections 2024-09-13T23:43:29.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/982934715 wait complete. 2024-09-13T23:43:29.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 Processor -- start 2024-09-13T23:43:29.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 -- start start 2024-09-13T23:43:29.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 0x55e2419e1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e241786000 con 0x55e241b40c00 2024-09-13T23:43:29.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a2ef75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 0x55e2419e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a2ef75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 0x55e2419e1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51674/0 (socket says 172.21.15.167:51674) 2024-09-13T23:43:29.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a2ef75700 1 -- 172.21.15.167:0/957313106 learned_addr learned my addr 172.21.15.167:0/957313106 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a2ef75700 1 -- 172.21.15.167:0/957313106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24170ad00 con 0x55e241b40c00 2024-09-13T23:43:29.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a2ef75700 1 --2- 172.21.15.167:0/957313106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 0x55e2419e1180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55e241b54120 tx=0x55e241b4c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a14740700 1 -- 172.21.15.167:0/957313106 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e241b422c0 con 0x55e241b40c00 2024-09-13T23:43:29.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a14740700 1 -- 172.21.15.167:0/957313106 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e241ac2600 con 0x55e241b40c00 2024-09-13T23:43:29.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/957313106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e24170a340 con 0x55e241b40c00 2024-09-13T23:43:29.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a14740700 1 -- 172.21.15.167:0/957313106 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e241b42420 con 0x55e241b40c00 2024-09-13T23:43:29.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.787+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/957313106 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e24170bd40 con 0x55e241b40c00 2024-09-13T23:43:29.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.791+0000 7f5a14740700 1 -- 172.21.15.167:0/957313106 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e2417f3b80 con 0x55e241b40c00 2024-09-13T23:43:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.791+0000 7f5a14740700 1 --2- 172.21.15.167:0/957313106 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241bca000 0x55e241bd0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.791+0000 7f5a14740700 1 -- 172.21.15.167:0/957313106 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e24175f6c0 con 0x55e241b40c00 2024-09-13T23:43:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.791+0000 7f5a30fbdf40 5 note: GC not initialized 2024-09-13T23:43:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.791+0000 7f5a2ff77700 1 --2- 172.21.15.167:0/957313106 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241bca000 0x55e241bd0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.791+0000 7f5a2ff77700 1 --2- 172.21.15.167:0/957313106 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241bca000 0x55e241bd0000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55e241ad5e60 tx=0x55e241a52d80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.791+0000 7f5a11f3b700 20 reqs_thread_entry: start 2024-09-13T23:43:29.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.795+0000 7f5a30fbdf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:29.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.795+0000 7f5a30fbdf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:29.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.795+0000 7f5a0ff37700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:43:59 2024 2024-09-13T23:43:29.803 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:29.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.795+0000 7f5a0f715700 20 reqs_thread_entry: start 2024-09-13T23:43:29.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.795+0000 7f5a30fbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.795+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b41800 con 0x55e241a99400 2024-09-13T23:43:29.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.795+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.795+0000 7f5a30fbdf40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:29.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.795+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b41c00 con 0x55e241b2e000 2024-09-13T23:43:29.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.795+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 155+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.795+0000 7f5a16744700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:29.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.795+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94430253244416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241bca400 con 0x55e241a99400 2024-09-13T23:43:29.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.799+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94430253244416] v33'91 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.799+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94430251801344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241bca800 con 0x55e241a99c00 2024-09-13T23:43:29.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.803+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94430251801344] v33'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.803+0000 7f5a30fbdf40 20 remove_watcher() i=0 2024-09-13T23:43:29.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.803+0000 7f5a30fbdf40 2 removed watcher, disabling cache 2024-09-13T23:43:29.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.803+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94430251802496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241bcac00 con 0x55e241a99400 2024-09-13T23:43:29.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.803+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94430251802496] v33'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241a99400 2024-09-13T23:43:29.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.803+0000 7f5a30fbdf40 20 remove_watcher() i=1 2024-09-13T23:43:29.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.803+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94430251803648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241bcb000 con 0x55e241a99c00 2024-09-13T23:43:29.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.807+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94430251803648] v33'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.807+0000 7f5a30fbdf40 20 remove_watcher() i=2 2024-09-13T23:43:29.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.807+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94430251804800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241bcb400 con 0x55e241b2e000 2024-09-13T23:43:29.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.807+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94430251804800] v33'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.807+0000 7f5a30fbdf40 20 remove_watcher() i=3 2024-09-13T23:43:29.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.807+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94430251805952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241bcb800 con 0x55e241a99c00 2024-09-13T23:43:29.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.811+0000 7f5a2f776700 1 -- 172.21.15.167:0/3599133295 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94430251805952] v33'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f200 con 0x55e241a99c00 2024-09-13T23:43:29.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.811+0000 7f5a30fbdf40 20 remove_watcher() i=4 2024-09-13T23:43:29.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.811+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94430251807104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241bcbc00 con 0x55e241b2e800 2024-09-13T23:43:29.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.815+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94430251807104] v33'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.815+0000 7f5a30fbdf40 20 remove_watcher() i=5 2024-09-13T23:43:29.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.815+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94430251808256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b40000 con 0x55e241b2e800 2024-09-13T23:43:29.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.815+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94430251808256] v33'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1f440 con 0x55e241b2e800 2024-09-13T23:43:29.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.815+0000 7f5a30fbdf40 20 remove_watcher() i=6 2024-09-13T23:43:29.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.815+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94430251809408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e241b40400 con 0x55e241b2e000 2024-09-13T23:43:29.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a2ff77700 1 -- 172.21.15.167:0/3599133295 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94430251809408] v33'86 uv84 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e241b1eb40 con 0x55e241b2e000 2024-09-13T23:43:29.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 20 remove_watcher() i=7 2024-09-13T23:43:29.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e241a99c00 msgr2=0x55e2419e0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e241a99c00 0x55e2419e0100 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e241a99400 msgr2=0x55e2419dfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e241a99400 0x55e2419dfb80 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e241b2e000 msgr2=0x55e2419e0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e241b2e000 0x55e2419e0c00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e241b2e800 msgr2=0x55e2419e0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e241b2e800 0x55e2419e0680 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241a99000 msgr2=0x55e2419df600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241a99000 0x55e2419df600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55e2419d9920 tx=0x55e2417fd800 comp rx=0 tx=0).stop 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 msgr2=0x55e2419deb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/3599133295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9800 0x55e2419deb00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55e241a9ae40 tx=0x55e241a52fc0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 reap_dead start 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 shutdown_connections 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 >> 172.21.15.167:0/3599133295 conn(0x55e2416f9400 msgr2=0x55e2417f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a2ef75700 1 -- 172.21.15.167:0/3599133295 reap_dead start 2024-09-13T23:43:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 shutdown_connections 2024-09-13T23:43:29.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/3599133295 wait complete. 2024-09-13T23:43:29.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/957313106 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241bca000 msgr2=0x55e241bd0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/957313106 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241bca000 0x55e241bd0000 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55e241ad5e60 tx=0x55e241a52d80 comp rx=0 tx=0).stop 2024-09-13T23:43:29.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/957313106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 msgr2=0x55e2419e1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/957313106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 0x55e2419e1180 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55e241b54120 tx=0x55e241b4c300 comp rx=0 tx=0).stop 2024-09-13T23:43:29.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/957313106 shutdown_connections 2024-09-13T23:43:29.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/957313106 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241bca000 0x55e241bd0000 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/957313106 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e241b40c00 0x55e2419e1180 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/957313106 >> 172.21.15.167:0/957313106 conn(0x55e241b40800 msgr2=0x55e2417fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/957313106 shutdown_connections 2024-09-13T23:43:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.819+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/957313106 wait complete. 2024-09-13T23:43:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.823+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/1339317086 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241a98000 msgr2=0x55e2419df080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.823+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/1339317086 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241a98000 0x55e2419df080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x55e242570b10 tx=0x55e241a52de0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.823+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/1339317086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 msgr2=0x55e2419de580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.823+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/1339317086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 0x55e2419de580 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x55e2424bbec0 tx=0x55e2417fda40 comp rx=0 tx=0).stop 2024-09-13T23:43:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.823+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/1339317086 shutdown_connections 2024-09-13T23:43:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.823+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/1339317086 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e241a98000 0x55e2419df080 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.823+0000 7f5a30fbdf40 1 --2- 172.21.15.167:0/1339317086 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e2416f9000 0x55e2419de580 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.823+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/1339317086 >> 172.21.15.167:0/1339317086 conn(0x55e2416f8c00 msgr2=0x55e2417f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.823+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/1339317086 shutdown_connections 2024-09-13T23:43:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.823+0000 7f5a30fbdf40 1 -- 172.21.15.167:0/1339317086 wait complete. 2024-09-13T23:43:29.834 INFO:teuthology.orchestra.run.smithi167.stdout:[] 2024-09-13T23:43:29.836 DEBUG:tasks.util.rgw: json result: [] 2024-09-13T23:43:29.871 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:29.872 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-09-13T23:43:29.872 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-09-13T23:43:29.872 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-09-13T23:43:29.913 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:29.913 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:29.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.903+0000 7ff22d331f40 1 Processor -- start 2024-09-13T23:43:29.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.903+0000 7ff22d331f40 1 -- start start 2024-09-13T23:43:29.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.903+0000 7ff22d331f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x558191525400 0x55819180a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.903+0000 7ff22d331f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5581915e9500 con 0x558191525000 2024-09-13T23:43:29.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.903+0000 7ff22d331f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5581915b2a00 con 0x558191525400 2024-09-13T23:43:29.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22b2e9700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x558191525400 0x55819180a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22baea700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x558191525000 0x558191624800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45394/0 (socket says 172.21.15.167:45394) 2024-09-13T23:43:29.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22baea700 1 -- 172.21.15.167:0/3654056756 learned_addr learned my addr 172.21.15.167:0/3654056756 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:29.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22b2e9700 1 -- 172.21.15.167:0/3654056756 >> v1:172.21.15.12:6789/0 conn(0x558191525000 legacy=0x558191624800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22b2e9700 1 -- 172.21.15.167:0/3654056756 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558191537d40 con 0x558191525400 2024-09-13T23:43:29.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22b2e9700 1 --2- 172.21.15.167:0/3654056756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525400 0x55819180a580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x558191805920 tx=0x558191629920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1002a701fef902 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22aae8700 1 -- 172.21.15.167:0/3654056756 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581915c94a0 con 0x558191525400 2024-09-13T23:43:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22aae8700 1 -- 172.21.15.167:0/3654056756 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581915e9800 con 0x558191525400 2024-09-13T23:43:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22aae8700 1 -- 172.21.15.167:0/3654056756 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581915c9600 con 0x558191525400 2024-09-13T23:43:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22d331f40 1 -- 172.21.15.167:0/3654056756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525400 msgr2=0x55819180a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22d331f40 1 --2- 172.21.15.167:0/3654056756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525400 0x55819180a580 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x558191805920 tx=0x558191629920 comp rx=0 tx=0).stop 2024-09-13T23:43:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22d331f40 1 -- 172.21.15.167:0/3654056756 shutdown_connections 2024-09-13T23:43:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22d331f40 1 --2- 172.21.15.167:0/3654056756 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525400 0x55819180a580 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22d331f40 1 -- 172.21.15.167:0/3654056756 >> 172.21.15.167:0/3654056756 conn(0x558191524c00 msgr2=0x558191624000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22d331f40 1 -- 172.21.15.167:0/3654056756 shutdown_connections 2024-09-13T23:43:29.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22d331f40 1 -- 172.21.15.167:0/3654056756 wait complete. 2024-09-13T23:43:29.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22d331f40 1 Processor -- start 2024-09-13T23:43:29.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22d331f40 1 -- start start 2024-09-13T23:43:29.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22d331f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 0x55819180ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22d331f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581915b2500 con 0x558191525000 2024-09-13T23:43:29.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22baea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 0x55819180ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.907+0000 7ff22baea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 0x55819180ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51692/0 (socket says 172.21.15.167:51692) 2024-09-13T23:43:29.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22baea700 1 -- 172.21.15.167:0/2891871833 learned_addr learned my addr 172.21.15.167:0/2891871833 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22baea700 1 -- 172.21.15.167:0/2891871833 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558191537ba0 con 0x558191525000 2024-09-13T23:43:29.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22baea700 1 --2- 172.21.15.167:0/2891871833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 0x55819180ab00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55819237d8c0 tx=0x558191629b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f78fa6d2156844 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22a2e7700 1 -- 172.21.15.167:0/2891871833 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581915c94a0 con 0x558191525000 2024-09-13T23:43:29.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22a2e7700 1 -- 172.21.15.167:0/2891871833 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581915e9b00 con 0x558191525000 2024-09-13T23:43:29.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22a2e7700 1 -- 172.21.15.167:0/2891871833 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581915c9600 con 0x558191525000 2024-09-13T23:43:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22d331f40 1 -- 172.21.15.167:0/2891871833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 msgr2=0x55819180ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22d331f40 1 --2- 172.21.15.167:0/2891871833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 0x55819180ab00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55819237d8c0 tx=0x558191629b60 comp rx=0 tx=0).stop 2024-09-13T23:43:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22d331f40 1 -- 172.21.15.167:0/2891871833 shutdown_connections 2024-09-13T23:43:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22d331f40 1 --2- 172.21.15.167:0/2891871833 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 0x55819180ab00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22d331f40 1 -- 172.21.15.167:0/2891871833 >> 172.21.15.167:0/2891871833 conn(0x558191524c00 msgr2=0x558191625000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22d331f40 1 -- 172.21.15.167:0/2891871833 shutdown_connections 2024-09-13T23:43:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22d331f40 1 -- 172.21.15.167:0/2891871833 wait complete. 2024-09-13T23:43:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22d331f40 1 Processor -- start 2024-09-13T23:43:29.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22d331f40 1 -- start start 2024-09-13T23:43:29.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22d331f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 0x55819180ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22d331f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581915b23c0 con 0x558191525000 2024-09-13T23:43:29.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22baea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 0x55819180ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22baea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 0x55819180ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51704/0 (socket says 172.21.15.167:51704) 2024-09-13T23:43:29.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22baea700 1 -- 172.21.15.167:0/3408216301 learned_addr learned my addr 172.21.15.167:0/3408216301 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.911+0000 7ff22baea700 1 -- 172.21.15.167:0/3408216301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558191537a00 con 0x558191525000 2024-09-13T23:43:29.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22baea700 1 --2- 172.21.15.167:0/3408216301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 0x55819180ab00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55819237db00 tx=0x558191629a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff228ae4700 1 -- 172.21.15.167:0/3408216301 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581915c98c0 con 0x558191525000 2024-09-13T23:43:29.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff228ae4700 1 -- 172.21.15.167:0/3408216301 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581915e9c80 con 0x558191525000 2024-09-13T23:43:29.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff228ae4700 1 -- 172.21.15.167:0/3408216301 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581915c94a0 con 0x558191525000 2024-09-13T23:43:29.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22d331f40 1 -- 172.21.15.167:0/3408216301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558191537860 con 0x558191525000 2024-09-13T23:43:29.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22d331f40 1 -- 172.21.15.167:0/3408216301 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581915376c0 con 0x558191525000 2024-09-13T23:43:29.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff228ae4700 1 -- 172.21.15.167:0/3408216301 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55819161eb00 con 0x558191525000 2024-09-13T23:43:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22d331f40 1 Processor -- start 2024-09-13T23:43:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff228ae4700 1 --2- 172.21.15.167:0/3408216301 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5581918c0000 0x55819180b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff228ae4700 1 -- 172.21.15.167:0/3408216301 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55819158b180 con 0x558191525000 2024-09-13T23:43:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22d331f40 1 -- start start 2024-09-13T23:43:29.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22c2eb700 1 --2- 172.21.15.167:0/3408216301 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5581918c0000 0x55819180b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22d331f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 0x55819180b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22d331f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581915b2000 con 0x5581918c0400 2024-09-13T23:43:29.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22c2eb700 1 --2- 172.21.15.167:0/3408216301 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5581918c0000 0x55819180b080 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55819239c990 tx=0x55819187ade0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22baea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 0x55819180b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22baea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 0x55819180b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51710/0 (socket says 172.21.15.167:51710) 2024-09-13T23:43:29.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22baea700 1 -- 172.21.15.167:0/440337356 learned_addr learned my addr 172.21.15.167:0/440337356 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22baea700 1 -- 172.21.15.167:0/440337356 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558191536000 con 0x5581918c0400 2024-09-13T23:43:29.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.915+0000 7ff22baea700 1 --2- 172.21.15.167:0/440337356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 0x55819180b600 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55819239d0b0 tx=0x55819187a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b075ebe159cd55e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff2262df700 1 -- 172.21.15.167:0/440337356 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581915c9600 con 0x5581918c0400 2024-09-13T23:43:29.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff2262df700 1 -- 172.21.15.167:0/440337356 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581915e9c80 con 0x5581918c0400 2024-09-13T23:43:29.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff2262df700 1 -- 172.21.15.167:0/440337356 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581915c9a20 con 0x5581918c0400 2024-09-13T23:43:29.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22d331f40 1 -- 172.21.15.167:0/440337356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 msgr2=0x55819180b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22d331f40 1 --2- 172.21.15.167:0/440337356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 0x55819180b600 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55819239d0b0 tx=0x55819187a240 comp rx=0 tx=0).stop 2024-09-13T23:43:29.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22d331f40 1 -- 172.21.15.167:0/440337356 shutdown_connections 2024-09-13T23:43:29.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22d331f40 1 --2- 172.21.15.167:0/440337356 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 0x55819180b600 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22d331f40 1 -- 172.21.15.167:0/440337356 >> 172.21.15.167:0/440337356 conn(0x558191525800 msgr2=0x558191625800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22d331f40 1 -- 172.21.15.167:0/440337356 shutdown_connections 2024-09-13T23:43:29.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22d331f40 1 -- 172.21.15.167:0/440337356 wait complete. 2024-09-13T23:43:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22d331f40 1 Processor -- start 2024-09-13T23:43:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22d331f40 1 -- start start 2024-09-13T23:43:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22d331f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 0x55819180b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22d331f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819154a640 con 0x5581918c0400 2024-09-13T23:43:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22baea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 0x55819180b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22baea700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 0x55819180b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51722/0 (socket says 172.21.15.167:51722) 2024-09-13T23:43:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 learned_addr learned my addr 172.21.15.167:0/2622629689 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558191536ea0 con 0x5581918c0400 2024-09-13T23:43:29.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.919+0000 7ff22baea700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 0x55819180b600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55819239c360 tx=0x55819187afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff224adc700 1 -- 172.21.15.167:0/2622629689 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581915c9600 con 0x5581918c0400 2024-09-13T23:43:29.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff224adc700 1 -- 172.21.15.167:0/2622629689 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581918e0000 con 0x5581918c0400 2024-09-13T23:43:29.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558191536820 con 0x5581918c0400 2024-09-13T23:43:29.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff224adc700 1 -- 172.21.15.167:0/2622629689 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581915c9a20 con 0x5581918c0400 2024-09-13T23:43:29.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581915369c0 con 0x5581918c0400 2024-09-13T23:43:29.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff224adc700 1 -- 172.21.15.167:0/2622629689 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55819161edc0 con 0x5581918c0400 2024-09-13T23:43:29.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff224adc700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558191930000 0x55819180c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff224adc700 1 -- 172.21.15.167:0/2622629689 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55819158b340 con 0x5581918c0400 2024-09-13T23:43:29.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff22b2e9700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558191930000 0x55819180c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff22d331f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff22d331f40 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5581918c0800 0x55819180bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5581918c0c00 con 0x5581918c0800 2024-09-13T23:43:29.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff22c2eb700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5581918c0800 0x55819180bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff22b2e9700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558191930000 0x55819180c680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5581918bbbf0 tx=0x558191629800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.923+0000 7ff22c2eb700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5581918c0800 0x55819180bb80 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 20 realm 2024-09-13T23:43:29.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5581918c1000 con 0x5581918c0800 2024-09-13T23:43:29.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:29.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191930400 con 0x5581918c0800 2024-09-13T23:43:29.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x558191930800 0x55819180c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191930c00 con 0x558191930800 2024-09-13T23:43:29.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22baea700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x558191930800 0x55819180c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22baea700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x558191930800 0x55819180c100 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:29.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.927+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5581918c0c00 con 0x558191930800 2024-09-13T23:43:29.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22d331f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:29.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22d331f40 20 searching for the correct realm 2024-09-13T23:43:29.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22d331f40 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5581918c1000 0x55819180cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c1000 2024-09-13T23:43:29.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22b2e9700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5581918c1000 0x55819180cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22b2e9700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5581918c1000 0x55819180cc00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930800 2024-09-13T23:43:29.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22baea700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x558191930c00 0x558191966000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930c00 2024-09-13T23:43:29.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22c2eb700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x558191930c00 0x558191966000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22c2eb700 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x558191930c00 0x558191966000 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:29.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.931+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c0800 2024-09-13T23:43:29.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c0800 2024-09-13T23:43:29.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c1000 2024-09-13T23:43:29.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c0800 2024-09-13T23:43:29.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930800 2024-09-13T23:43:29.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930800 2024-09-13T23:43:29.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c0800 2024-09-13T23:43:29.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930c00 2024-09-13T23:43:29.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:29.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.935+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c0800 2024-09-13T23:43:29.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.939+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.939+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930c00 2024-09-13T23:43:29.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.939+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:29.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.939+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930800 2024-09-13T23:43:29.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.939+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.939+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c1000 2024-09-13T23:43:29.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.939+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.939+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c0800 2024-09-13T23:43:29.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.939+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.939+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930800 2024-09-13T23:43:29.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.939+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.939+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c1000 2024-09-13T23:43:29.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c0800 2024-09-13T23:43:29.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c0800 2024-09-13T23:43:29.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c0800 2024-09-13T23:43:29.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c1000 2024-09-13T23:43:29.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930c00 2024-09-13T23:43:29.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:29.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c0800 2024-09-13T23:43:29.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.943+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930c00 2024-09-13T23:43:29.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:29.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c1000 2024-09-13T23:43:29.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930c00 2024-09-13T23:43:29.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:29.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930800 2024-09-13T23:43:29.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930800 2024-09-13T23:43:29.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c1000 2024-09-13T23:43:29.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930800 2024-09-13T23:43:29.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.947+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930800 2024-09-13T23:43:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931400 con 0x5581918c0800 2024-09-13T23:43:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931800 con 0x558191930800 2024-09-13T23:43:29.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:29.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931c00 con 0x558191930800 2024-09-13T23:43:29.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:29.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 zone default found 2024-09-13T23:43:29.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 4 Realm: () 2024-09-13T23:43:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 zonegroup default 2024-09-13T23:43:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196e000 con 0x558191930c00 2024-09-13T23:43:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196e400 con 0x5581918c1000 2024-09-13T23:43:29.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.951+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196e800 con 0x5581918c1000 2024-09-13T23:43:29.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.955+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.955+0000 7ff22d331f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:29.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.955+0000 7ff22d331f40 20 started sync module instance, tier type = 2024-09-13T23:43:29.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.955+0000 7ff22d331f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:29.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.955+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196ec00 con 0x558191930800 2024-09-13T23:43:29.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.955+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.955+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94014980233984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196f000 con 0x558191930800 2024-09-13T23:43:29.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.955+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196f400 con 0x5581918c0800 2024-09-13T23:43:29.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.959+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.959+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94014980233984] v33'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.959+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94014980235136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196f800 con 0x5581918c0800 2024-09-13T23:43:29.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.959+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196ec00 con 0x558191930800 2024-09-13T23:43:29.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.963+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94014980235136] v33'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.963+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.963+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94014980236288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x558191930800 2024-09-13T23:43:29.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.963+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931800 con 0x558191930c00 2024-09-13T23:43:29.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.963+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:29.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.963+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94014980237440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931c00 con 0x558191930c00 2024-09-13T23:43:29.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.963+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191930400 con 0x558191930800 2024-09-13T23:43:29.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.963+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94014980236288] v33'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.967+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94014980237440] v33'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:29.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.967+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.967+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94014980238592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931400 con 0x558191930800 2024-09-13T23:43:29.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.967+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5581918c0c00 con 0x5581918c1000 2024-09-13T23:43:29.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.967+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.967+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94014980239744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5581918c1800 con 0x5581918c1000 2024-09-13T23:43:29.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.967+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196e400 con 0x5581918c1000 2024-09-13T23:43:29.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.971+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94014980238592] v33'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:29.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.971+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94014980239744] v33'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.971+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.971+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94014980240896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196e800 con 0x5581918c1000 2024-09-13T23:43:29.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.971+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196f400 con 0x558191930c00 2024-09-13T23:43:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'87 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94014980242048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196f800 con 0x558191930c00 2024-09-13T23:43:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94014980240896] v33'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22d331f40 20 add_watcher() i=0 2024-09-13T23:43:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22d331f40 20 add_watcher() i=1 2024-09-13T23:43:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22d331f40 20 add_watcher() i=2 2024-09-13T23:43:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22d331f40 20 add_watcher() i=3 2024-09-13T23:43:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22d331f40 20 add_watcher() i=4 2024-09-13T23:43:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22d331f40 20 add_watcher() i=5 2024-09-13T23:43:29.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22d331f40 20 add_watcher() i=6 2024-09-13T23:43:29.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94014980242048] v33'88 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:29.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22d331f40 20 add_watcher() i=7 2024-09-13T23:43:29.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22d331f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:29.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55819196e000 con 0x5581918c0800 2024-09-13T23:43:29.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.975+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94014981750784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931800 con 0x5581918c0800 2024-09-13T23:43:29.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94014981750784] v33'92 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff212ab8700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:29.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff22d331f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:29.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff22d331f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:29.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff22d331f40 1 Processor -- start 2024-09-13T23:43:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff22d331f40 1 -- start start 2024-09-13T23:43:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff22d331f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0c00 0x55819180d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff22d331f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55819154a780 con 0x5581918c0c00 2024-09-13T23:43:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff22b2e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0c00 0x55819180d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff22b2e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0c00 0x55819180d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51738/0 (socket says 172.21.15.167:51738) 2024-09-13T23:43:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff22b2e9700 1 -- 172.21.15.167:0/1859436377 learned_addr learned my addr 172.21.15.167:0/1859436377 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff22b2e9700 1 -- 172.21.15.167:0/1859436377 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558191536b60 con 0x5581918c0c00 2024-09-13T23:43:29.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff22b2e9700 1 --2- 172.21.15.167:0/1859436377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0c00 0x55819180d180 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x5581918f9dd0 tx=0x558191629860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f019664066ae5890 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff2122b7700 1 -- 172.21.15.167:0/1859436377 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581915c9760 con 0x5581918c0c00 2024-09-13T23:43:29.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff2122b7700 1 -- 172.21.15.167:0/1859436377 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581918e0900 con 0x5581918c0c00 2024-09-13T23:43:29.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.979+0000 7ff2122b7700 1 -- 172.21.15.167:0/1859436377 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558191972160 con 0x5581918c0c00 2024-09-13T23:43:29.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 -- 172.21.15.167:0/1859436377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0c00 msgr2=0x55819180d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:29.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 --2- 172.21.15.167:0/1859436377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0c00 0x55819180d180 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x5581918f9dd0 tx=0x558191629860 comp rx=0 tx=0).stop 2024-09-13T23:43:29.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 -- 172.21.15.167:0/1859436377 shutdown_connections 2024-09-13T23:43:29.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 --2- 172.21.15.167:0/1859436377 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0c00 0x55819180d180 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:29.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 -- 172.21.15.167:0/1859436377 >> 172.21.15.167:0/1859436377 conn(0x558191931c00 msgr2=0x558191626000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:29.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 -- 172.21.15.167:0/1859436377 shutdown_connections 2024-09-13T23:43:29.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 -- 172.21.15.167:0/1859436377 wait complete. 2024-09-13T23:43:29.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 Processor -- start 2024-09-13T23:43:29.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 -- start start 2024-09-13T23:43:29.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525400 0x55819180d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581915b2000 con 0x558191525400 2024-09-13T23:43:29.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22b2e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525400 0x55819180d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22b2e9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525400 0x55819180d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51748/0 (socket says 172.21.15.167:51748) 2024-09-13T23:43:29.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22b2e9700 1 -- 172.21.15.167:0/452299443 learned_addr learned my addr 172.21.15.167:0/452299443 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:29.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22b2e9700 1 -- 172.21.15.167:0/452299443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558191536d00 con 0x558191525400 2024-09-13T23:43:29.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22b2e9700 1 --2- 172.21.15.167:0/452299443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525400 0x55819180d700 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x558191981200 tx=0x558191982300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff210ab4700 1 -- 172.21.15.167:0/452299443 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5581919722c0 con 0x558191525400 2024-09-13T23:43:29.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff210ab4700 1 -- 172.21.15.167:0/452299443 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581918e0600 con 0x558191525400 2024-09-13T23:43:29.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 -- 172.21.15.167:0/452299443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558191536340 con 0x558191525400 2024-09-13T23:43:29.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff210ab4700 1 -- 172.21.15.167:0/452299443 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558191972420 con 0x558191525400 2024-09-13T23:43:29.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.983+0000 7ff22d331f40 1 -- 172.21.15.167:0/452299443 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558191537d40 con 0x558191525400 2024-09-13T23:43:29.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff210ab4700 1 -- 172.21.15.167:0/452299443 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55819161fb80 con 0x558191525400 2024-09-13T23:43:29.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff210ab4700 1 --2- 172.21.15.167:0/452299443 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55819196fc00 0x55819180a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:29.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff210ab4700 1 -- 172.21.15.167:0/452299443 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55819158b6c0 con 0x558191525400 2024-09-13T23:43:29.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff22c2eb700 1 --2- 172.21.15.167:0/452299443 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55819196fc00 0x55819180a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:29.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff22d331f40 5 note: GC not initialized 2024-09-13T23:43:29.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff22c2eb700 1 --2- 172.21.15.167:0/452299443 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55819196fc00 0x55819180a580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5581918bb530 tx=0x55819187ad20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:29.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff20e2af700 20 reqs_thread_entry: start 2024-09-13T23:43:29.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff22d331f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:29.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff22d331f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:29.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff20c2ab700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:00 2024 2024-09-13T23:43:29.998 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:29.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff20ba89700 20 reqs_thread_entry: start 2024-09-13T23:43:29.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff22d331f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:29.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.987+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191a10800 con 0x5581918c0800 2024-09-13T23:43:29.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.991+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:29.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.991+0000 7ff22d331f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:29.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.991+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191a10c00 con 0x558191930c00 2024-09-13T23:43:30.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.991+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv2 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:30.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.991+0000 7ff212ab8700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:30.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.991+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94014981750784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191a11000 con 0x5581918c0800 2024-09-13T23:43:30.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.995+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94014981750784] v33'93 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:30.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.995+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94014980233984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191a11400 con 0x558191930800 2024-09-13T23:43:30.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.995+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94014980233984] v33'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:30.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.995+0000 7ff22d331f40 20 remove_watcher() i=0 2024-09-13T23:43:30.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.995+0000 7ff22d331f40 2 removed watcher, disabling cache 2024-09-13T23:43:30.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.995+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94014980235136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191a11800 con 0x5581918c0800 2024-09-13T23:43:30.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.999+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94014980235136] v33'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191952b40 con 0x5581918c0800 2024-09-13T23:43:30.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.999+0000 7ff22d331f40 20 remove_watcher() i=1 2024-09-13T23:43:30.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.999+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94014980236288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191a11c00 con 0x558191930800 2024-09-13T23:43:30.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.999+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94014980236288] v33'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:30.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.999+0000 7ff22d331f40 20 remove_watcher() i=2 2024-09-13T23:43:30.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:29.999+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94014980237440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191a11400 con 0x558191930c00 2024-09-13T23:43:30.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.003+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94014980237440] v33'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:30.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.003+0000 7ff22d331f40 20 remove_watcher() i=3 2024-09-13T23:43:30.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.003+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94014980238592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191a11c00 con 0x558191930800 2024-09-13T23:43:30.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.007+0000 7ff22baea700 1 -- 172.21.15.167:0/2622629689 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94014980238592] v33'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953200 con 0x558191930800 2024-09-13T23:43:30.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.007+0000 7ff22d331f40 20 remove_watcher() i=4 2024-09-13T23:43:30.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.007+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94014980239744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931000 con 0x5581918c1000 2024-09-13T23:43:30.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.007+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94014980239744] v33'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:30.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.007+0000 7ff22d331f40 20 remove_watcher() i=5 2024-09-13T23:43:30.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.007+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94014980240896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191930400 con 0x5581918c1000 2024-09-13T23:43:30.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94014980240896] v33'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191953440 con 0x5581918c1000 2024-09-13T23:43:30.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 20 remove_watcher() i=6 2024-09-13T23:43:30.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94014980242048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x558191931400 con 0x558191930c00 2024-09-13T23:43:30.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22c2eb700 1 -- 172.21.15.167:0/2622629689 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94014980242048] v33'89 uv87 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x558191952b40 con 0x558191930c00 2024-09-13T23:43:30.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 20 remove_watcher() i=7 2024-09-13T23:43:30.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x558191930800 msgr2=0x55819180c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x558191930800 0x55819180c100 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5581918c0800 msgr2=0x55819180bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5581918c0800 0x55819180bb80 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x558191930c00 msgr2=0x558191966000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x558191930c00 0x558191966000 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5581918c1000 msgr2=0x55819180cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5581918c1000 0x55819180cc00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558191930000 msgr2=0x55819180c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558191930000 0x55819180c680 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x5581918bbbf0 tx=0x558191629800 comp rx=0 tx=0).stop 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 msgr2=0x55819180b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22b2e9700 1 -- 172.21.15.167:0/2622629689 reap_dead start 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 0x55819180b600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x55819239c360 tx=0x55819187afc0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 shutdown_connections 2024-09-13T23:43:30.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 --2- 172.21.15.167:0/2622629689 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5581918c0400 0x55819180b600 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 >> 172.21.15.167:0/2622629689 conn(0x558191525800 msgr2=0x558191625800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 shutdown_connections 2024-09-13T23:43:30.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.011+0000 7ff22d331f40 1 -- 172.21.15.167:0/2622629689 wait complete. 2024-09-13T23:43:30.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 -- 172.21.15.167:0/452299443 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55819196fc00 msgr2=0x55819180a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 --2- 172.21.15.167:0/452299443 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55819196fc00 0x55819180a580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x5581918bb530 tx=0x55819187ad20 comp rx=0 tx=0).stop 2024-09-13T23:43:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 -- 172.21.15.167:0/452299443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525400 msgr2=0x55819180d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 --2- 172.21.15.167:0/452299443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525400 0x55819180d700 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x558191981200 tx=0x558191982300 comp rx=0 tx=0).stop 2024-09-13T23:43:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 -- 172.21.15.167:0/452299443 shutdown_connections 2024-09-13T23:43:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 --2- 172.21.15.167:0/452299443 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55819196fc00 0x55819180a580 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 --2- 172.21.15.167:0/452299443 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525400 0x55819180d700 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 -- 172.21.15.167:0/452299443 >> 172.21.15.167:0/452299443 conn(0x558191931c00 msgr2=0x558191626000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 -- 172.21.15.167:0/452299443 shutdown_connections 2024-09-13T23:43:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 -- 172.21.15.167:0/452299443 wait complete. 2024-09-13T23:43:30.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 -- 172.21.15.167:0/3408216301 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5581918c0000 msgr2=0x55819180b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 --2- 172.21.15.167:0/3408216301 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5581918c0000 0x55819180b080 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55819239c990 tx=0x55819187ade0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 -- 172.21.15.167:0/3408216301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 msgr2=0x55819180ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 --2- 172.21.15.167:0/3408216301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 0x55819180ab00 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x55819237db00 tx=0x558191629a40 comp rx=0 tx=0).stop 2024-09-13T23:43:30.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 -- 172.21.15.167:0/3408216301 shutdown_connections 2024-09-13T23:43:30.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 --2- 172.21.15.167:0/3408216301 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5581918c0000 0x55819180b080 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 --2- 172.21.15.167:0/3408216301 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558191525000 0x55819180ab00 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 -- 172.21.15.167:0/3408216301 >> 172.21.15.167:0/3408216301 conn(0x558191524c00 msgr2=0x558191625000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 -- 172.21.15.167:0/3408216301 shutdown_connections 2024-09-13T23:43:30.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.015+0000 7ff22d331f40 1 -- 172.21.15.167:0/3408216301 wait complete. 2024-09-13T23:43:30.029 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-09-13T23:43:30.029 INFO:teuthology.orchestra.run.smithi167.stdout: "myfoo" 2024-09-13T23:43:30.029 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-09-13T23:43:30.029 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-09-13T23:43:30.033 INFO:tasks.radosgw_admin:RL: list_buckets foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:30.033 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-09-13T23:43:30.034 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-09-13T23:43:30.034 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-09-13T23:43:30.108 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:30.109 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:30.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.099+0000 7ff9c22c4f40 1 Processor -- start 2024-09-13T23:43:30.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.099+0000 7ff9c22c4f40 1 -- start start 2024-09-13T23:43:30.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.099+0000 7ff9c22c4f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x555d67123000 0x555d67408580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.099+0000 7ff9c22c4f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x555d671e7500 con 0x555d67123400 2024-09-13T23:43:30.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.099+0000 7ff9c22c4f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x555d671b0a00 con 0x555d67123000 2024-09-13T23:43:30.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.099+0000 7ff9c0a7d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x555d67123000 0x555d67408580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.099+0000 7ff9c027c700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x555d67123400 0x555d67222800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45400/0 (socket says 172.21.15.167:45400) 2024-09-13T23:43:30.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.099+0000 7ff9c027c700 1 -- 172.21.15.167:0/3364241473 learned_addr learned my addr 172.21.15.167:0/3364241473 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:30.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9bfa7b700 1 -- 172.21.15.167:0/3364241473 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093632393 0 0) 0x555d6727e000 con 0x555d67123400 2024-09-13T23:43:30.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9bfa7b700 1 -- 172.21.15.167:0/3364241473 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x555d671e7680 con 0x555d67123400 2024-09-13T23:43:30.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/3364241473 >> v1:172.21.15.12:6789/0 conn(0x555d67123400 legacy=0x555d67222800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/3364241473 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d67135d40 con 0x555d67123000 2024-09-13T23:43:30.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9c0a7d700 1 --2- 172.21.15.167:0/3364241473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x555d67403a70 tx=0x555d67227980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fdbcc882f3461e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9bfa7b700 1 -- 172.21.15.167:0/3364241473 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d671c74a0 con 0x555d67123000 2024-09-13T23:43:30.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9bfa7b700 1 -- 172.21.15.167:0/3364241473 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d671e7500 con 0x555d67123000 2024-09-13T23:43:30.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9bfa7b700 1 -- 172.21.15.167:0/3364241473 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d671c7600 con 0x555d67123000 2024-09-13T23:43:30.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/3364241473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 msgr2=0x555d67408580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/3364241473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x555d67403a70 tx=0x555d67227980 comp rx=0 tx=0).stop 2024-09-13T23:43:30.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/3364241473 shutdown_connections 2024-09-13T23:43:30.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/3364241473 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/3364241473 >> 172.21.15.167:0/3364241473 conn(0x555d67122c00 msgr2=0x555d67222000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/3364241473 shutdown_connections 2024-09-13T23:43:30.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.103+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/3364241473 wait complete. 2024-09-13T23:43:30.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c22c4f40 1 Processor -- start 2024-09-13T23:43:30.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c22c4f40 1 -- start start 2024-09-13T23:43:30.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c22c4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c22c4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d671b0500 con 0x555d67123000 2024-09-13T23:43:30.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c0a7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c0a7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51766/0 (socket says 172.21.15.167:51766) 2024-09-13T23:43:30.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/3418804610 learned_addr learned my addr 172.21.15.167:0/3418804610 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/3418804610 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d67135ba0 con 0x555d67123000 2024-09-13T23:43:30.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c0a7d700 1 --2- 172.21.15.167:0/3418804610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x555d67eede00 tx=0x555d67227800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563d840d685c7376 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9bf27a700 1 -- 172.21.15.167:0/3418804610 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d671c7760 con 0x555d67123000 2024-09-13T23:43:30.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9bf27a700 1 -- 172.21.15.167:0/3418804610 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d671e7980 con 0x555d67123000 2024-09-13T23:43:30.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9bf27a700 1 -- 172.21.15.167:0/3418804610 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d671c74a0 con 0x555d67123000 2024-09-13T23:43:30.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/3418804610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 msgr2=0x555d67408580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/3418804610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x555d67eede00 tx=0x555d67227800 comp rx=0 tx=0).stop 2024-09-13T23:43:30.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/3418804610 shutdown_connections 2024-09-13T23:43:30.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.107+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/3418804610 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/3418804610 >> 172.21.15.167:0/3418804610 conn(0x555d67122c00 msgr2=0x555d67223000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/3418804610 shutdown_connections 2024-09-13T23:43:30.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/3418804610 wait complete. 2024-09-13T23:43:30.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c22c4f40 1 Processor -- start 2024-09-13T23:43:30.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c22c4f40 1 -- start start 2024-09-13T23:43:30.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c22c4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c22c4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d671b03c0 con 0x555d67123000 2024-09-13T23:43:30.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c0a7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c0a7d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51780/0 (socket says 172.21.15.167:51780) 2024-09-13T23:43:30.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/750984211 learned_addr learned my addr 172.21.15.167:0/750984211 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/750984211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d67135a00 con 0x555d67123000 2024-09-13T23:43:30.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c0a7d700 1 --2- 172.21.15.167:0/750984211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x555d67eedef0 tx=0x555d67227980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9bda77700 1 -- 172.21.15.167:0/750984211 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d671c7600 con 0x555d67123000 2024-09-13T23:43:30.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9bda77700 1 -- 172.21.15.167:0/750984211 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d671e7b00 con 0x555d67123000 2024-09-13T23:43:30.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/750984211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555d67135860 con 0x555d67123000 2024-09-13T23:43:30.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9bda77700 1 -- 172.21.15.167:0/750984211 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d671c78c0 con 0x555d67123000 2024-09-13T23:43:30.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.111+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/750984211 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d671356c0 con 0x555d67123000 2024-09-13T23:43:30.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 Processor -- start 2024-09-13T23:43:30.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 -- start start 2024-09-13T23:43:30.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 0x555d67408b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d671b0000 con 0x555d67123800 2024-09-13T23:43:30.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c127e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 0x555d67408b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c127e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 0x555d67408b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51784/0 (socket says 172.21.15.167:51784) 2024-09-13T23:43:30.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c127e700 1 -- 172.21.15.167:0/511152995 learned_addr learned my addr 172.21.15.167:0/511152995 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9bda77700 1 -- 172.21.15.167:0/750984211 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x555d6721cb00 con 0x555d67123000 2024-09-13T23:43:30.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c127e700 1 -- 172.21.15.167:0/511152995 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d67134000 con 0x555d67123800 2024-09-13T23:43:30.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9bda77700 1 --2- 172.21.15.167:0/750984211 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d674d6000 0x555d67409080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9bda77700 1 -- 172.21.15.167:0/750984211 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x555d67189180 con 0x555d67123000 2024-09-13T23:43:30.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c0a7d700 1 --2- 172.21.15.167:0/750984211 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d674d6000 0x555d67409080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c127e700 1 --2- 172.21.15.167:0/511152995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 0x555d67408b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x555d6743d6b0 tx=0x555d6748e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc980b90d21faa2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9bb272700 1 -- 172.21.15.167:0/511152995 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d671c7600 con 0x555d67123800 2024-09-13T23:43:30.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9bb272700 1 -- 172.21.15.167:0/511152995 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d671e7b00 con 0x555d67123800 2024-09-13T23:43:30.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9bb272700 1 -- 172.21.15.167:0/511152995 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d671c78c0 con 0x555d67123800 2024-09-13T23:43:30.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/511152995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 msgr2=0x555d67408b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/511152995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 0x555d67408b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x555d6743d6b0 tx=0x555d6748e660 comp rx=0 tx=0).stop 2024-09-13T23:43:30.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c0a7d700 1 --2- 172.21.15.167:0/750984211 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d674d6000 0x555d67409080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x555d6743d5f0 tx=0x555d6748ec60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/511152995 shutdown_connections 2024-09-13T23:43:30.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/511152995 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 0x555d67408b00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/511152995 >> 172.21.15.167:0/511152995 conn(0x555d67123400 msgr2=0x555d67223800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/511152995 shutdown_connections 2024-09-13T23:43:30.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/511152995 wait complete. 2024-09-13T23:43:30.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 Processor -- start 2024-09-13T23:43:30.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 -- start start 2024-09-13T23:43:30.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 0x555d67408b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.115+0000 7ff9c22c4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d67148640 con 0x555d67123800 2024-09-13T23:43:30.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9c127e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 0x555d67408b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9c127e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 0x555d67408b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51792/0 (socket says 172.21.15.167:51792) 2024-09-13T23:43:30.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 learned_addr learned my addr 172.21.15.167:0/1503033680 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d67134ea0 con 0x555d67123800 2024-09-13T23:43:30.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9c127e700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 0x555d67408b00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x555d6743c2a0 tx=0x555d6748e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9b9a6f700 1 -- 172.21.15.167:0/1503033680 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d671c7760 con 0x555d67123800 2024-09-13T23:43:30.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9b9a6f700 1 -- 172.21.15.167:0/1503033680 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d6748a000 con 0x555d67123800 2024-09-13T23:43:30.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555d67134820 con 0x555d67123800 2024-09-13T23:43:30.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9b9a6f700 1 -- 172.21.15.167:0/1503033680 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d671c7600 con 0x555d67123800 2024-09-13T23:43:30.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d671349c0 con 0x555d67123800 2024-09-13T23:43:30.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9b9a6f700 1 -- 172.21.15.167:0/1503033680 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x555d6721cdc0 con 0x555d67123800 2024-09-13T23:43:30.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9b9a6f700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d674d6c00 0x555d67409600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9b9a6f700 1 -- 172.21.15.167:0/1503033680 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x555d67189340 con 0x555d67123800 2024-09-13T23:43:30.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9c22c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x555d674d7000 0x555d67409b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.119+0000 7ff9c027c700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d674d6c00 0x555d67409600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7400 con 0x555d674d7000 2024-09-13T23:43:30.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c127e700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x555d674d7000 0x555d67409b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c027c700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d674d6c00 0x555d67409600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x555d67eec030 tx=0x555d67570060 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c127e700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x555d674d7000 0x555d67409b80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 20 realm 2024-09-13T23:43:30.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7000 2024-09-13T23:43:30.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:30.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7c00 con 0x555d674d7000 2024-09-13T23:43:30.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x555d674d7400 0x555d6740a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7400 2024-09-13T23:43:30.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c0a7d700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x555d674d7400 0x555d6740a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.123+0000 7ff9c0a7d700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x555d674d7400 0x555d6740a100 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c22c4f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:30.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c22c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7c00 con 0x555d674d7400 2024-09-13T23:43:30.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c22c4f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:30.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c22c4f40 20 searching for the correct realm 2024-09-13T23:43:30.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x555d67452000 0x555d6740a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67452400 con 0x555d67452000 2024-09-13T23:43:30.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c027c700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x555d67452000 0x555d6740a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c027c700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x555d67452000 0x555d6740a680 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67452400 con 0x555d674d7400 2024-09-13T23:43:30.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c0a7d700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x555d67452400 0x555d6740ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452400 2024-09-13T23:43:30.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.127+0000 7ff9c127e700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x555d67452400 0x555d6740ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c127e700 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x555d67452400 0x555d6740ac00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7000 2024-09-13T23:43:30.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7000 2024-09-13T23:43:30.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452000 2024-09-13T23:43:30.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7000 2024-09-13T23:43:30.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7400 2024-09-13T23:43:30.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.131+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7400 2024-09-13T23:43:30.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7000 2024-09-13T23:43:30.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452400 2024-09-13T23:43:30.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7000 2024-09-13T23:43:30.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452400 2024-09-13T23:43:30.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7400 2024-09-13T23:43:30.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452000 2024-09-13T23:43:30.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7000 2024-09-13T23:43:30.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.135+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7400 2024-09-13T23:43:30.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.139+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.139+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452000 2024-09-13T23:43:30.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.139+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.139+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7000 2024-09-13T23:43:30.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.139+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.139+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7000 2024-09-13T23:43:30.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.139+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.139+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7000 2024-09-13T23:43:30.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.139+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.139+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452000 2024-09-13T23:43:30.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.139+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.139+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452400 2024-09-13T23:43:30.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7000 2024-09-13T23:43:30.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452400 2024-09-13T23:43:30.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452000 2024-09-13T23:43:30.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452400 2024-09-13T23:43:30.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7400 2024-09-13T23:43:30.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7400 2024-09-13T23:43:30.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.143+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452000 2024-09-13T23:43:30.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7400 2024-09-13T23:43:30.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7400 2024-09-13T23:43:30.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c22c4f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:30.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c22c4f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:30.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c22c4f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:30.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c22c4f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:30.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c22c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67452800 con 0x555d674d7000 2024-09-13T23:43:30.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c22c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c22c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67452c00 con 0x555d674d7400 2024-09-13T23:43:30.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c22c4f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:30.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c22c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.147+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67453000 con 0x555d674d7400 2024-09-13T23:43:30.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:30.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 20 zone default found 2024-09-13T23:43:30.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 4 Realm: () 2024-09-13T23:43:30.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:30.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:30.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:30.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 20 zonegroup default 2024-09-13T23:43:30.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67453400 con 0x555d67452400 2024-09-13T23:43:30.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67453800 con 0x555d67452000 2024-09-13T23:43:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67453c00 con 0x555d67452000 2024-09-13T23:43:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 20 started sync module instance, tier type = 2024-09-13T23:43:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.151+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67453800 con 0x555d674d7400 2024-09-13T23:43:30.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.155+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.155+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93859652565760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67453c00 con 0x555d674d7400 2024-09-13T23:43:30.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.155+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67452c00 con 0x555d674d7000 2024-09-13T23:43:30.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.159+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.159+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93859652566912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67453000 con 0x555d674d7000 2024-09-13T23:43:30.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.159+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67453800 con 0x555d674d7400 2024-09-13T23:43:30.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.159+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 93859652565760] v33'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.159+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93859652566912] v33'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.159+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.159+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93859652568064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7c00 con 0x555d674d7400 2024-09-13T23:43:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.159+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d67452400 2024-09-13T23:43:30.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.163+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 93859652568064] v33'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.163+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.163+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93859652569216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67452800 con 0x555d67452400 2024-09-13T23:43:30.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.163+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67453400 con 0x555d674d7400 2024-09-13T23:43:30.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.167+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 93859652569216] v33'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.167+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.167+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93859652570368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67452c00 con 0x555d674d7400 2024-09-13T23:43:30.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.167+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67453000 con 0x555d67452000 2024-09-13T23:43:30.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.167+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.167+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93859652571520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452000 2024-09-13T23:43:30.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.167+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93859652570368] v33'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.167+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454400 con 0x555d67452000 2024-09-13T23:43:30.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.171+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93859652571520] v33'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.171+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.171+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93859652572672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454800 con 0x555d67452000 2024-09-13T23:43:30.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.171+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454c00 con 0x555d67452400 2024-09-13T23:43:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93859652572672] v33'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'90 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93859652573824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67455000 con 0x555d67452400 2024-09-13T23:43:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c22c4f40 20 add_watcher() i=0 2024-09-13T23:43:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c22c4f40 20 add_watcher() i=1 2024-09-13T23:43:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c22c4f40 20 add_watcher() i=2 2024-09-13T23:43:30.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c22c4f40 20 add_watcher() i=3 2024-09-13T23:43:30.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c22c4f40 20 add_watcher() i=4 2024-09-13T23:43:30.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c22c4f40 20 add_watcher() i=5 2024-09-13T23:43:30.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c22c4f40 20 add_watcher() i=6 2024-09-13T23:43:30.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 93859652573824] v33'91 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c22c4f40 20 add_watcher() i=7 2024-09-13T23:43:30.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c22c4f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:30.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67455400 con 0x555d674d7000 2024-09-13T23:43:30.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.175+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93859654230016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67455800 con 0x555d674d7000 2024-09-13T23:43:30.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93859654230016] v33'94 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9a7a4b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:30.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9c22c4f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:30.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9c22c4f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:30.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9c22c4f40 1 Processor -- start 2024-09-13T23:43:30.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9c22c4f40 1 -- start start 2024-09-13T23:43:30.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9c22c4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 0x555d6740b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9c22c4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d67148780 con 0x555d67454c00 2024-09-13T23:43:30.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9c027c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 0x555d6740b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9c027c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 0x555d6740b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51806/0 (socket says 172.21.15.167:51806) 2024-09-13T23:43:30.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9c027c700 1 -- 172.21.15.167:0/1788305559 learned_addr learned my addr 172.21.15.167:0/1788305559 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9c027c700 1 -- 172.21.15.167:0/1788305559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d67134b60 con 0x555d67454c00 2024-09-13T23:43:30.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9c027c700 1 --2- 172.21.15.167:0/1788305559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 0x555d6740b180 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x555d6759e0c0 tx=0x555d675701e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4f112662a30094 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.179+0000 7ff9a724a700 1 -- 172.21.15.167:0/1788305559 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d675a4000 con 0x555d67454c00 2024-09-13T23:43:30.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9a724a700 1 -- 172.21.15.167:0/1788305559 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d6748a480 con 0x555d67454c00 2024-09-13T23:43:30.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9a724a700 1 -- 172.21.15.167:0/1788305559 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d675a4160 con 0x555d67454c00 2024-09-13T23:43:30.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1788305559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 msgr2=0x555d6740b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/1788305559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 0x555d6740b180 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x555d6759e0c0 tx=0x555d675701e0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1788305559 shutdown_connections 2024-09-13T23:43:30.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/1788305559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 0x555d6740b180 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1788305559 >> 172.21.15.167:0/1788305559 conn(0x555d67455c00 msgr2=0x555d67224000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1788305559 shutdown_connections 2024-09-13T23:43:30.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1788305559 wait complete. 2024-09-13T23:43:30.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c22c4f40 1 Processor -- start 2024-09-13T23:43:30.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c22c4f40 1 -- start start 2024-09-13T23:43:30.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c22c4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 0x555d6740b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c22c4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d671b0000 con 0x555d67454c00 2024-09-13T23:43:30.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c027c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 0x555d6740b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c027c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 0x555d6740b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51822/0 (socket says 172.21.15.167:51822) 2024-09-13T23:43:30.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c027c700 1 -- 172.21.15.167:0/540565703 learned_addr learned my addr 172.21.15.167:0/540565703 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c027c700 1 -- 172.21.15.167:0/540565703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d67134d00 con 0x555d67454c00 2024-09-13T23:43:30.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9c027c700 1 --2- 172.21.15.167:0/540565703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 0x555d6740b180 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x555d6759f380 tx=0x555d675700c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9a5a47700 1 -- 172.21.15.167:0/540565703 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d675a42c0 con 0x555d67454c00 2024-09-13T23:43:30.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.183+0000 7ff9a5a47700 1 -- 172.21.15.167:0/540565703 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d6748a600 con 0x555d67454c00 2024-09-13T23:43:30.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.187+0000 7ff9a5a47700 1 -- 172.21.15.167:0/540565703 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555d675a4000 con 0x555d67454c00 2024-09-13T23:43:30.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.187+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/540565703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555d67134340 con 0x555d67454c00 2024-09-13T23:43:30.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.187+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/540565703 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d67134b60 con 0x555d67454c00 2024-09-13T23:43:30.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.187+0000 7ff9a5a47700 1 -- 172.21.15.167:0/540565703 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x555d6721db80 con 0x555d67454c00 2024-09-13T23:43:30.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.187+0000 7ff9a5a47700 1 --2- 172.21.15.167:0/540565703 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d67628000 0x555d6762e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.187+0000 7ff9a5a47700 1 -- 172.21.15.167:0/540565703 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x555d67189500 con 0x555d67454c00 2024-09-13T23:43:30.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.187+0000 7ff9c22c4f40 5 note: GC not initialized 2024-09-13T23:43:30.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.187+0000 7ff9c127e700 1 --2- 172.21.15.167:0/540565703 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d67628000 0x555d6762e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.187+0000 7ff9c127e700 1 --2- 172.21.15.167:0/540565703 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d67628000 0x555d6762e000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x555d67515770 tx=0x555d6748f4a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.187+0000 7ff9a3242700 20 reqs_thread_entry: start 2024-09-13T23:43:30.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.187+0000 7ff9c22c4f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:30.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.187+0000 7ff9c22c4f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:30.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9a123e700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:00 2024 2024-09-13T23:43:30.199 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:30.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452000 2024-09-13T23:43:30.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9a0a1c700 20 reqs_thread_entry: start 2024-09-13T23:43:30.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(62 [pgnls start_epoch 33 out=127b] v33'1 uv1 ondisk = 1) v8 ==== 144+0+127 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7400 2024-09-13T23:43:30.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(63 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7000 2024-09-13T23:43:30.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 [pgnls start_epoch 33 out=49b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7000 2024-09-13T23:43:30.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(65 [pgnls start_epoch 33 out=49b] v33'11 uv11 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452400 2024-09-13T23:43:30.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(66 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.191+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7400 2024-09-13T23:43:30.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.195+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.195+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7000 2024-09-13T23:43:30.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.195+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(68 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.195+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452000 2024-09-13T23:43:30.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.195+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(69 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.195+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452400 2024-09-13T23:43:30.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.195+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(70 [pgnls start_epoch 33 out=49b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.195+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7400 2024-09-13T23:43:30.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.195+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(71 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.195+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7000 2024-09-13T23:43:30.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(72 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452000 2024-09-13T23:43:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(73 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7000 2024-09-13T23:43:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(74 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452000 2024-09-13T23:43:30.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(75 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452400 2024-09-13T23:43:30.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(76 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452000 2024-09-13T23:43:30.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(77 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7000 2024-09-13T23:43:30.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(78 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7400 2024-09-13T23:43:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(79 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.199+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452400 2024-09-13T23:43:30.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(80 [pgnls start_epoch 33 out=70b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+70 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7400 2024-09-13T23:43:30.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(81 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7000 2024-09-13T23:43:30.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(82 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452000 2024-09-13T23:43:30.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(83 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452400 2024-09-13T23:43:30.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(84 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7000 2024-09-13T23:43:30.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(85 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7000 2024-09-13T23:43:30.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(86 [pgnls start_epoch 33 out=49b] v33'1 uv1 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.203+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452000 2024-09-13T23:43:30.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(87 [pgnls start_epoch 33 out=49b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452000 2024-09-13T23:43:30.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(88 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452400 2024-09-13T23:43:30.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(89 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7000 2024-09-13T23:43:30.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 28 ==== osd_op_reply(90 [pgnls start_epoch 33 out=49b] v33'1 uv1 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d674d7400 2024-09-13T23:43:30.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(91 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452000 2024-09-13T23:43:30.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(92 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454000 con 0x555d67452400 2024-09-13T23:43:30.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 18 ==== osd_op_reply(93 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c22c4f40 20 RGWRados::pool_iterate: got .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 2024-09-13T23:43:30.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.207+0000 7ff9c22c4f40 20 RGWRados::pool_iterate: got myfoo 2024-09-13T23:43:30.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.211+0000 7ff9a7a4b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:30.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.211+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93859654230016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67454400 con 0x555d674d7000 2024-09-13T23:43:30.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.211+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 93859654230016] v33'95 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.211+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93859652565760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67452800 con 0x555d674d7400 2024-09-13T23:43:30.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.215+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 93859652565760] v33'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.215+0000 7ff9c22c4f40 20 remove_watcher() i=0 2024-09-13T23:43:30.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.215+0000 7ff9c22c4f40 2 removed watcher, disabling cache 2024-09-13T23:43:30.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.215+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93859652566912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67453000 con 0x555d674d7000 2024-09-13T23:43:30.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.219+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.1 v2:172.21.15.167:6800/3875661733 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 93859652566912] v33'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d674d7000 2024-09-13T23:43:30.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.219+0000 7ff9c22c4f40 20 remove_watcher() i=1 2024-09-13T23:43:30.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.219+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93859652568064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d674d7800 con 0x555d674d7400 2024-09-13T23:43:30.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.219+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 93859652568064] v33'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.223+0000 7ff9c22c4f40 20 remove_watcher() i=2 2024-09-13T23:43:30.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.223+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93859652569216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67628400 con 0x555d67452400 2024-09-13T23:43:30.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.223+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 19 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 93859652569216] v33'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.223+0000 7ff9c22c4f40 20 remove_watcher() i=3 2024-09-13T23:43:30.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.223+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93859652570368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67628800 con 0x555d674d7400 2024-09-13T23:43:30.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.227+0000 7ff9c0a7d700 1 -- 172.21.15.167:0/1503033680 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 93859652570368] v33'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583200 con 0x555d674d7400 2024-09-13T23:43:30.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.227+0000 7ff9c22c4f40 20 remove_watcher() i=4 2024-09-13T23:43:30.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.227+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93859652571520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67628c00 con 0x555d67452000 2024-09-13T23:43:30.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.227+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 93859652571520] v33'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.227+0000 7ff9c22c4f40 20 remove_watcher() i=5 2024-09-13T23:43:30.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.227+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93859652572672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67629000 con 0x555d67452000 2024-09-13T23:43:30.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.231+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 93859652572672] v33'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67583440 con 0x555d67452000 2024-09-13T23:43:30.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.231+0000 7ff9c22c4f40 20 remove_watcher() i=6 2024-09-13T23:43:30.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.231+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93859652573824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555d67629400 con 0x555d67452400 2024-09-13T23:43:30.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c127e700 1 -- 172.21.15.167:0/1503033680 <== osd.2 v2:172.21.15.167:6808/1180369139 20 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 93859652573824] v33'92 uv90 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555d67582b40 con 0x555d67452400 2024-09-13T23:43:30.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 20 remove_watcher() i=7 2024-09-13T23:43:30.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x555d674d7400 msgr2=0x555d6740a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x555d674d7400 0x555d6740a100 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x555d674d7000 msgr2=0x555d67409b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x555d674d7000 0x555d67409b80 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x555d67452400 msgr2=0x555d6740ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x555d67452400 0x555d6740ac00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x555d67452000 msgr2=0x555d6740a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x555d67452000 0x555d6740a680 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d674d6c00 msgr2=0x555d67409600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d674d6c00 0x555d67409600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x555d67eec030 tx=0x555d67570060 comp rx=0 tx=0).stop 2024-09-13T23:43:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 msgr2=0x555d67408b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 0x555d67408b00 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x555d6743c2a0 tx=0x555d6748e000 comp rx=0 tx=0).stop 2024-09-13T23:43:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c027c700 1 -- 172.21.15.167:0/1503033680 reap_dead start 2024-09-13T23:43:30.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 shutdown_connections 2024-09-13T23:43:30.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/1503033680 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123800 0x555d67408b00 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 >> 172.21.15.167:0/1503033680 conn(0x555d67123400 msgr2=0x555d67223800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 shutdown_connections 2024-09-13T23:43:30.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/1503033680 wait complete. 2024-09-13T23:43:30.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/540565703 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d67628000 msgr2=0x555d6762e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/540565703 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d67628000 0x555d6762e000 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x555d67515770 tx=0x555d6748f4a0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/540565703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 msgr2=0x555d6740b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/540565703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 0x555d6740b180 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x555d6759f380 tx=0x555d675700c0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/540565703 shutdown_connections 2024-09-13T23:43:30.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/540565703 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d67628000 0x555d6762e000 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/540565703 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67454c00 0x555d6740b180 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/540565703 >> 172.21.15.167:0/540565703 conn(0x555d67455c00 msgr2=0x555d67224000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/540565703 shutdown_connections 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/540565703 wait complete. 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/750984211 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d674d6000 msgr2=0x555d67409080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/750984211 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d674d6000 0x555d67409080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x555d6743d5f0 tx=0x555d6748ec60 comp rx=0 tx=0).stop 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/750984211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 msgr2=0x555d67408580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/750984211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x555d67eedef0 tx=0x555d67227980 comp rx=0 tx=0).stop 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/750984211 shutdown_connections 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/750984211 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555d674d6000 0x555d67409080 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 --2- 172.21.15.167:0/750984211 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555d67123000 0x555d67408580 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/750984211 >> 172.21.15.167:0/750984211 conn(0x555d67122c00 msgr2=0x555d67223000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/750984211 shutdown_connections 2024-09-13T23:43:30.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.235+0000 7ff9c22c4f40 1 -- 172.21.15.167:0/750984211 wait complete. 2024-09-13T23:43:30.248 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-09-13T23:43:30.248 INFO:teuthology.orchestra.run.smithi167.stdout: "myfoo" 2024-09-13T23:43:30.248 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-09-13T23:43:30.251 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-09-13T23:43:30.272 INFO:tasks.radosgw_admin:RL: create_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:30.291 INFO:tasks.radosgw_admin:RL: create_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:30.309 INFO:tasks.radosgw_admin:RL: create_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:30.353 INFO:tasks.radosgw_admin:RL: create_bucket myfoo5 foo bytes_out=0 bytes_in=237 failed=True 2024-09-13T23:43:30.430 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:30.495 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:30.562 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:30.562 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-09-13T23:43:30.562 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-09-13T23:43:30.563 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-09-13T23:43:30.601 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:30.601 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:30.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.591+0000 7f96f5e66f40 1 Processor -- start 2024-09-13T23:43:30.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.591+0000 7f96f5e66f40 1 -- start start 2024-09-13T23:43:30.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.591+0000 7f96f5e66f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55706b21d400 0x55706b502580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.591+0000 7f96f5e66f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55706b2e1500 con 0x55706b21d000 2024-09-13T23:43:30.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.591+0000 7f96f5e66f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55706b2aaa00 con 0x55706b21d400 2024-09-13T23:43:30.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f3e1e700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55706b21d400 0x55706b502580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f461f700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55706b21d000 0x55706b31c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45414/0 (socket says 172.21.15.167:45414) 2024-09-13T23:43:30.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f461f700 1 -- 172.21.15.167:0/1638065405 learned_addr learned my addr 172.21.15.167:0/1638065405 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:30.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f361d700 1 -- 172.21.15.167:0/1638065405 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193524882 0 0) 0x55706b378000 con 0x55706b21d000 2024-09-13T23:43:30.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f361d700 1 -- 172.21.15.167:0/1638065405 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55706b2e1680 con 0x55706b21d000 2024-09-13T23:43:30.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f3e1e700 1 -- 172.21.15.167:0/1638065405 >> v1:172.21.15.12:6789/0 conn(0x55706b21d000 legacy=0x55706b31c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f3e1e700 1 -- 172.21.15.167:0/1638065405 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55706b22fd40 con 0x55706b21d400 2024-09-13T23:43:30.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f3e1e700 1 --2- 172.21.15.167:0/1638065405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d400 0x55706b502580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55706b4fd920 tx=0x55706b321980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49648467a70b47d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f361d700 1 -- 172.21.15.167:0/1638065405 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b2c14a0 con 0x55706b21d400 2024-09-13T23:43:30.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f361d700 1 -- 172.21.15.167:0/1638065405 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55706b2e1680 con 0x55706b21d400 2024-09-13T23:43:30.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f361d700 1 -- 172.21.15.167:0/1638065405 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b2c1600 con 0x55706b21d400 2024-09-13T23:43:30.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1638065405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d400 msgr2=0x55706b502580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/1638065405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d400 0x55706b502580 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55706b4fd920 tx=0x55706b321980 comp rx=0 tx=0).stop 2024-09-13T23:43:30.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1638065405 shutdown_connections 2024-09-13T23:43:30.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/1638065405 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d400 0x55706b502580 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1638065405 >> 172.21.15.167:0/1638065405 conn(0x55706b21cc00 msgr2=0x55706b31c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1638065405 shutdown_connections 2024-09-13T23:43:30.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1638065405 wait complete. 2024-09-13T23:43:30.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f5e66f40 1 Processor -- start 2024-09-13T23:43:30.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f5e66f40 1 -- start start 2024-09-13T23:43:30.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f5e66f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 0x55706b502580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f5e66f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706b2aa500 con 0x55706b21d000 2024-09-13T23:43:30.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f461f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 0x55706b502580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f461f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 0x55706b502580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51838/0 (socket says 172.21.15.167:51838) 2024-09-13T23:43:30.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.595+0000 7f96f461f700 1 -- 172.21.15.167:0/1739255710 learned_addr learned my addr 172.21.15.167:0/1739255710 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f461f700 1 -- 172.21.15.167:0/1739255710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55706b22fba0 con 0x55706b21d000 2024-09-13T23:43:30.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f461f700 1 --2- 172.21.15.167:0/1739255710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 0x55706b502580 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55706bfe7d70 tx=0x55706b321b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be786c6a2e4ca6da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f2e1c700 1 -- 172.21.15.167:0/1739255710 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b2c14a0 con 0x55706b21d000 2024-09-13T23:43:30.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f2e1c700 1 -- 172.21.15.167:0/1739255710 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55706b2e1b00 con 0x55706b21d000 2024-09-13T23:43:30.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f2e1c700 1 -- 172.21.15.167:0/1739255710 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b2c1600 con 0x55706b21d000 2024-09-13T23:43:30.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1739255710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 msgr2=0x55706b502580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/1739255710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 0x55706b502580 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x55706bfe7d70 tx=0x55706b321b60 comp rx=0 tx=0).stop 2024-09-13T23:43:30.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1739255710 shutdown_connections 2024-09-13T23:43:30.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/1739255710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 0x55706b502580 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1739255710 >> 172.21.15.167:0/1739255710 conn(0x55706b21cc00 msgr2=0x55706b31d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1739255710 shutdown_connections 2024-09-13T23:43:30.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1739255710 wait complete. 2024-09-13T23:43:30.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f5e66f40 1 Processor -- start 2024-09-13T23:43:30.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f5e66f40 1 -- start start 2024-09-13T23:43:30.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f5e66f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 0x55706b502580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f5e66f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706b2aa3c0 con 0x55706b21d000 2024-09-13T23:43:30.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f461f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 0x55706b502580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f461f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 0x55706b502580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51842/0 (socket says 172.21.15.167:51842) 2024-09-13T23:43:30.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f461f700 1 -- 172.21.15.167:0/3913665964 learned_addr learned my addr 172.21.15.167:0/3913665964 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f461f700 1 -- 172.21.15.167:0/3913665964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55706b22fa00 con 0x55706b21d000 2024-09-13T23:43:30.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.599+0000 7f96f461f700 1 --2- 172.21.15.167:0/3913665964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 0x55706b502580 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55706bfe7e00 tx=0x55706b321920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f1619700 1 -- 172.21.15.167:0/3913665964 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b2c18c0 con 0x55706b21d000 2024-09-13T23:43:30.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f1619700 1 -- 172.21.15.167:0/3913665964 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55706b2e1c80 con 0x55706b21d000 2024-09-13T23:43:30.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f5e66f40 1 -- 172.21.15.167:0/3913665964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55706b22f860 con 0x55706b21d000 2024-09-13T23:43:30.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f1619700 1 -- 172.21.15.167:0/3913665964 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b2c14a0 con 0x55706b21d000 2024-09-13T23:43:30.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f5e66f40 1 -- 172.21.15.167:0/3913665964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55706b22f6c0 con 0x55706b21d000 2024-09-13T23:43:30.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f5e66f40 1 Processor -- start 2024-09-13T23:43:30.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f5e66f40 1 -- start start 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f5e66f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 0x55706b502b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f5e66f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706b2aa000 con 0x55706b21d800 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f4e20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 0x55706b502b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f4e20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 0x55706b502b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51856/0 (socket says 172.21.15.167:51856) 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f4e20700 1 -- 172.21.15.167:0/1437813366 learned_addr learned my addr 172.21.15.167:0/1437813366 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f1619700 1 -- 172.21.15.167:0/3913665964 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55706b316b00 con 0x55706b21d000 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f4e20700 1 -- 172.21.15.167:0/1437813366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55706b22e000 con 0x55706b21d800 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f1619700 1 --2- 172.21.15.167:0/3913665964 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b5ce000 0x55706b503080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f1619700 1 -- 172.21.15.167:0/3913665964 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55706b283180 con 0x55706b21d000 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f4e20700 1 --2- 172.21.15.167:0/1437813366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 0x55706b502b00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55706b53cc30 tx=0x55706b56e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06e626c7c49c19f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96f461f700 1 --2- 172.21.15.167:0/3913665964 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b5ce000 0x55706b503080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96eee14700 1 -- 172.21.15.167:0/1437813366 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b2c18c0 con 0x55706b21d800 2024-09-13T23:43:30.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96eee14700 1 -- 172.21.15.167:0/1437813366 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55706b580000 con 0x55706b21d800 2024-09-13T23:43:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.603+0000 7f96eee14700 1 -- 172.21.15.167:0/1437813366 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b2c14a0 con 0x55706b21d800 2024-09-13T23:43:30.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1437813366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 msgr2=0x55706b502b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f461f700 1 --2- 172.21.15.167:0/3913665964 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b5ce000 0x55706b503080 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55706b53d080 tx=0x55706b56eea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/1437813366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 0x55706b502b00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x55706b53cc30 tx=0x55706b56e900 comp rx=0 tx=0).stop 2024-09-13T23:43:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1437813366 shutdown_connections 2024-09-13T23:43:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/1437813366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 0x55706b502b00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1437813366 >> 172.21.15.167:0/1437813366 conn(0x55706b21d400 msgr2=0x55706b31d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1437813366 shutdown_connections 2024-09-13T23:43:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 -- 172.21.15.167:0/1437813366 wait complete. 2024-09-13T23:43:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 Processor -- start 2024-09-13T23:43:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 -- start start 2024-09-13T23:43:30.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 0x55706b502b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706b242640 con 0x55706b21d800 2024-09-13T23:43:30.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f4e20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 0x55706b502b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f4e20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 0x55706b502b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51860/0 (socket says 172.21.15.167:51860) 2024-09-13T23:43:30.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 learned_addr learned my addr 172.21.15.167:0/2494398012 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55706b22eea0 con 0x55706b21d800 2024-09-13T23:43:30.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f4e20700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 0x55706b502b00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55706b53c330 tx=0x55706b56e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96ed611700 1 -- 172.21.15.167:0/2494398012 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b2c1b80 con 0x55706b21d800 2024-09-13T23:43:30.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96ed611700 1 -- 172.21.15.167:0/2494398012 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55706b580180 con 0x55706b21d800 2024-09-13T23:43:30.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55706b22e820 con 0x55706b21d800 2024-09-13T23:43:30.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96ed611700 1 -- 172.21.15.167:0/2494398012 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b2c18c0 con 0x55706b21d800 2024-09-13T23:43:30.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.607+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55706b22e9c0 con 0x55706b21d800 2024-09-13T23:43:30.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96ed611700 1 -- 172.21.15.167:0/2494398012 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55706b316dc0 con 0x55706b21d800 2024-09-13T23:43:30.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96ed611700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b5cec00 0x55706b503600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96ed611700 1 -- 172.21.15.167:0/2494398012 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55706b283340 con 0x55706b21d800 2024-09-13T23:43:30.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f3e1e700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b5cec00 0x55706b503600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55706b5cf000 0x55706b503b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf400 con 0x55706b5cf000 2024-09-13T23:43:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f4e20700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55706b5cf000 0x55706b503b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f3e1e700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b5cec00 0x55706b503600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55706b4fd920 tx=0x55706b321800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f4e20700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55706b5cf000 0x55706b503b80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 20 realm 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf000 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cfc00 con 0x55706b5cf000 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.611+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55706b5cf400 0x55706b504100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf400 2024-09-13T23:43:30.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f461f700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55706b5cf400 0x55706b504100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f461f700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55706b5cf400 0x55706b504100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f5e66f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:30.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cfc00 con 0x55706b5cf400 2024-09-13T23:43:30.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f5e66f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:30.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f5e66f40 20 searching for the correct realm 2024-09-13T23:43:30.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55706b538000 0x55706b504680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b538400 con 0x55706b538000 2024-09-13T23:43:30.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f3e1e700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55706b538000 0x55706b504680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f3e1e700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55706b538000 0x55706b504680 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.615+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b538400 con 0x55706b5cf400 2024-09-13T23:43:30.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f461f700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55706b538400 0x55706b504c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538400 2024-09-13T23:43:30.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f4e20700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55706b538400 0x55706b504c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f4e20700 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55706b538400 0x55706b504c00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf000 2024-09-13T23:43:30.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf000 2024-09-13T23:43:30.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538000 2024-09-13T23:43:30.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf000 2024-09-13T23:43:30.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.619+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf400 2024-09-13T23:43:30.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf400 2024-09-13T23:43:30.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf000 2024-09-13T23:43:30.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538400 2024-09-13T23:43:30.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf000 2024-09-13T23:43:30.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538400 2024-09-13T23:43:30.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf400 2024-09-13T23:43:30.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.623+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538000 2024-09-13T23:43:30.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf000 2024-09-13T23:43:30.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf400 2024-09-13T23:43:30.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538000 2024-09-13T23:43:30.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf000 2024-09-13T23:43:30.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf000 2024-09-13T23:43:30.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.627+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf000 2024-09-13T23:43:30.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538000 2024-09-13T23:43:30.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538400 2024-09-13T23:43:30.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf000 2024-09-13T23:43:30.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538400 2024-09-13T23:43:30.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538000 2024-09-13T23:43:30.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538400 2024-09-13T23:43:30.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.631+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf400 2024-09-13T23:43:30.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf400 2024-09-13T23:43:30.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538000 2024-09-13T23:43:30.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf400 2024-09-13T23:43:30.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf400 2024-09-13T23:43:30.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f5e66f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:30.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f5e66f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:30.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f5e66f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:30.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f5e66f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:30.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.635+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b538800 con 0x55706b5cf000 2024-09-13T23:43:30.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b538c00 con 0x55706b5cf400 2024-09-13T23:43:30.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:30.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539000 con 0x55706b5cf400 2024-09-13T23:43:30.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:30.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 20 zone default found 2024-09-13T23:43:30.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 4 Realm: () 2024-09-13T23:43:30.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:30.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:30.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:30.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 20 zonegroup default 2024-09-13T23:43:30.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539400 con 0x55706b538400 2024-09-13T23:43:30.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539800 con 0x55706b538000 2024-09-13T23:43:30.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.639+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539c00 con 0x55706b538000 2024-09-13T23:43:30.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.643+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.643+0000 7f96f5e66f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.643+0000 7f96f5e66f40 20 started sync module instance, tier type = 2024-09-13T23:43:30.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.643+0000 7f96f5e66f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:30.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.643+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539800 con 0x55706b5cf400 2024-09-13T23:43:30.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.643+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.643+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93941325077248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539c00 con 0x55706b5cf400 2024-09-13T23:43:30.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.643+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b538c00 con 0x55706b5cf000 2024-09-13T23:43:30.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.647+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.647+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 93941325077248] v33'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.647+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93941325078400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539000 con 0x55706b5cf000 2024-09-13T23:43:30.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.647+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539800 con 0x55706b5cf400 2024-09-13T23:43:30.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.647+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93941325078400] v33'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.651+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.651+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93941325079552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cfc00 con 0x55706b5cf400 2024-09-13T23:43:30.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.651+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b538400 2024-09-13T23:43:30.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.651+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.651+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93941325080704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b538800 con 0x55706b538400 2024-09-13T23:43:30.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.651+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539400 con 0x55706b5cf400 2024-09-13T23:43:30.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.651+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 93941325079552] v33'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.655+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 93941325080704] v33'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.655+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.655+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93941325081856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b538c00 con 0x55706b5cf400 2024-09-13T23:43:30.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.655+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539000 con 0x55706b538000 2024-09-13T23:43:30.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.659+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.659+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93941325083008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b53a000 con 0x55706b538000 2024-09-13T23:43:30.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.659+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b53a400 con 0x55706b538000 2024-09-13T23:43:30.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.659+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93941325081856] v33'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.659+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.659+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93941325084160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b53a800 con 0x55706b538000 2024-09-13T23:43:30.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.659+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b53ac00 con 0x55706b538400 2024-09-13T23:43:30.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.659+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93941325083008] v33'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93941325084160] v33'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'93 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93941325085312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b53b000 con 0x55706b538400 2024-09-13T23:43:30.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f5e66f40 20 add_watcher() i=0 2024-09-13T23:43:30.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f5e66f40 20 add_watcher() i=1 2024-09-13T23:43:30.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f5e66f40 20 add_watcher() i=2 2024-09-13T23:43:30.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f5e66f40 20 add_watcher() i=3 2024-09-13T23:43:30.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f5e66f40 20 add_watcher() i=4 2024-09-13T23:43:30.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f5e66f40 20 add_watcher() i=5 2024-09-13T23:43:30.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f5e66f40 20 add_watcher() i=6 2024-09-13T23:43:30.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 93941325085312] v33'94 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f5e66f40 20 add_watcher() i=7 2024-09-13T23:43:30.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f5e66f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:30.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b53b400 con 0x55706b5cf000 2024-09-13T23:43:30.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.663+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93941326749696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b53b800 con 0x55706b5cf000 2024-09-13T23:43:30.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93941326749696] v33'96 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96db5ed700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:30.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96f5e66f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:30.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96f5e66f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:30.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96f5e66f40 1 Processor -- start 2024-09-13T23:43:30.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96f5e66f40 1 -- start start 2024-09-13T23:43:30.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96f5e66f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 0x55706b505180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96f5e66f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706b242780 con 0x55706b53ac00 2024-09-13T23:43:30.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96f3e1e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 0x55706b505180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96f3e1e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 0x55706b505180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51866/0 (socket says 172.21.15.167:51866) 2024-09-13T23:43:30.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96f3e1e700 1 -- 172.21.15.167:0/586465750 learned_addr learned my addr 172.21.15.167:0/586465750 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96f3e1e700 1 -- 172.21.15.167:0/586465750 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55706b22eb60 con 0x55706b53ac00 2024-09-13T23:43:30.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.667+0000 7f96f3e1e700 1 --2- 172.21.15.167:0/586465750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 0x55706b505180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55706b61fd10 tx=0x55706b321860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce50f1630efd31 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96dadec700 1 -- 172.21.15.167:0/586465750 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b2c1760 con 0x55706b53ac00 2024-09-13T23:43:30.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96dadec700 1 -- 172.21.15.167:0/586465750 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55706b580600 con 0x55706b53ac00 2024-09-13T23:43:30.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96dadec700 1 -- 172.21.15.167:0/586465750 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b6a6000 con 0x55706b53ac00 2024-09-13T23:43:30.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f5e66f40 1 -- 172.21.15.167:0/586465750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 msgr2=0x55706b505180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/586465750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 0x55706b505180 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x55706b61fd10 tx=0x55706b321860 comp rx=0 tx=0).stop 2024-09-13T23:43:30.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f5e66f40 1 -- 172.21.15.167:0/586465750 shutdown_connections 2024-09-13T23:43:30.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/586465750 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 0x55706b505180 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f5e66f40 1 -- 172.21.15.167:0/586465750 >> 172.21.15.167:0/586465750 conn(0x55706b53bc00 msgr2=0x55706b31e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f5e66f40 1 -- 172.21.15.167:0/586465750 shutdown_connections 2024-09-13T23:43:30.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f5e66f40 1 -- 172.21.15.167:0/586465750 wait complete. 2024-09-13T23:43:30.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f5e66f40 1 Processor -- start 2024-09-13T23:43:30.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f5e66f40 1 -- start start 2024-09-13T23:43:30.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f5e66f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 0x55706b505180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f5e66f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55706b2aa000 con 0x55706b53ac00 2024-09-13T23:43:30.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f3e1e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 0x55706b505180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f3e1e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 0x55706b505180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51872/0 (socket says 172.21.15.167:51872) 2024-09-13T23:43:30.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f3e1e700 1 -- 172.21.15.167:0/921374824 learned_addr learned my addr 172.21.15.167:0/921374824 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f3e1e700 1 -- 172.21.15.167:0/921374824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55706b22ed00 con 0x55706b53ac00 2024-09-13T23:43:30.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f3e1e700 1 --2- 172.21.15.167:0/921374824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 0x55706b505180 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55706b69a1b0 tx=0x55706b69c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96d95e9700 1 -- 172.21.15.167:0/921374824 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b6a6160 con 0x55706b53ac00 2024-09-13T23:43:30.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96d95e9700 1 -- 172.21.15.167:0/921374824 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55706b580300 con 0x55706b53ac00 2024-09-13T23:43:30.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.671+0000 7f96f5e66f40 1 -- 172.21.15.167:0/921374824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55706b22e340 con 0x55706b53ac00 2024-09-13T23:43:30.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.675+0000 7f96d95e9700 1 -- 172.21.15.167:0/921374824 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55706b6a62c0 con 0x55706b53ac00 2024-09-13T23:43:30.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.675+0000 7f96f5e66f40 1 -- 172.21.15.167:0/921374824 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55706b22fd40 con 0x55706b53ac00 2024-09-13T23:43:30.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.675+0000 7f96d95e9700 1 -- 172.21.15.167:0/921374824 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55706b317b80 con 0x55706b53ac00 2024-09-13T23:43:30.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.675+0000 7f96d95e9700 1 --2- 172.21.15.167:0/921374824 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b71c000 0x55706b722000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.675+0000 7f96d95e9700 1 -- 172.21.15.167:0/921374824 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55706b283500 con 0x55706b53ac00 2024-09-13T23:43:30.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.675+0000 7f96f4e20700 1 --2- 172.21.15.167:0/921374824 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b71c000 0x55706b722000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.675+0000 7f96f5e66f40 5 note: GC not initialized 2024-09-13T23:43:30.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.675+0000 7f96f4e20700 1 --2- 172.21.15.167:0/921374824 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b71c000 0x55706b722000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55706b53d200 tx=0x55706b56f740 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.675+0000 7f96d6de4700 20 reqs_thread_entry: start 2024-09-13T23:43:30.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:30.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:30.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96d4de0700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:01 2024 2024-09-13T23:43:30.687 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:30.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96d45be700 20 reqs_thread_entry: start 2024-09-13T23:43:30.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b53a000 con 0x55706b538400 2024-09-13T23:43:30.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:30.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b53a400 con 0x55706b538000 2024-09-13T23:43:30.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:30.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-13T23:43:30.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b538800 con 0x55706b538400 2024-09-13T23:43:30.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:30.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539000 con 0x55706b538000 2024-09-13T23:43:30.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.679+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(65 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:30.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-13T23:43:30.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 4.a 4:517e2946:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf400 2024-09-13T23:43:30.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 4.8 4:10b47b19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b71c400 con 0x55706b538000 2024-09-13T23:43:30.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 4.2e 4:75bb53d2:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b71c800 con 0x55706b538000 2024-09-13T23:43:30.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 4.26 4:6424e2c9:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b71cc00 con 0x55706b5cf000 2024-09-13T23:43:30.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 4.c 4:32a29db1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b71d000 con 0x55706b5cf000 2024-09-13T23:43:30.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 4.28 4:162fa60a:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b71d400 con 0x55706b5cf400 2024-09-13T23:43:30.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 4.2c 4:355fcb42:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b71d800 con 0x55706b5cf000 2024-09-13T23:43:30.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 4.2f 4:f64e09ba:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b71dc00 con 0x55706b5cf000 2024-09-13T23:43:30.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:74 4.3a 4:5ee337b5:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cfc00 con 0x55706b538000 2024-09-13T23:43:30.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:75 4.2c 4:3559f32d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b5cf800 con 0x55706b5cf000 2024-09-13T23:43:30.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 4.1 4:8131d739:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539000 con 0x55706b538400 2024-09-13T23:43:30.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(67 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(66 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(68 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(71 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(69 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(72 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(74 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(70 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(73 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(75 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.9 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(76 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96db5ed700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:30.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.683+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93941326749696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539c00 con 0x55706b5cf000 2024-09-13T23:43:30.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.687+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 93941326749696] v33'97 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.687+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93941325077248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539800 con 0x55706b5cf400 2024-09-13T23:43:30.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.691+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 93941325077248] v33'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.691+0000 7f96f5e66f40 20 remove_watcher() i=0 2024-09-13T23:43:30.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.691+0000 7f96f5e66f40 2 removed watcher, disabling cache 2024-09-13T23:43:30.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.691+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93941325078400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b539400 con 0x55706b5cf000 2024-09-13T23:43:30.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.691+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 93941325078400] v33'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b5cf000 2024-09-13T23:43:30.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.691+0000 7f96f5e66f40 20 remove_watcher() i=1 2024-09-13T23:43:30.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.691+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93941325079552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b538c00 con 0x55706b5cf400 2024-09-13T23:43:30.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.695+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 93941325079552] v33'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.695+0000 7f96f5e66f40 20 remove_watcher() i=2 2024-09-13T23:43:30.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.695+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93941325080704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b71c400 con 0x55706b538400 2024-09-13T23:43:30.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.695+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 93941325080704] v33'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.695+0000 7f96f5e66f40 20 remove_watcher() i=3 2024-09-13T23:43:30.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.695+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93941325081856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b71c800 con 0x55706b5cf400 2024-09-13T23:43:30.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.699+0000 7f96f461f700 1 -- 172.21.15.167:0/2494398012 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 93941325081856] v33'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675200 con 0x55706b5cf400 2024-09-13T23:43:30.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.699+0000 7f96f5e66f40 20 remove_watcher() i=4 2024-09-13T23:43:30.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.699+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93941325083008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b53a400 con 0x55706b538000 2024-09-13T23:43:30.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.703+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 93941325083008] v33'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.703+0000 7f96f5e66f40 20 remove_watcher() i=5 2024-09-13T23:43:30.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.703+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93941325084160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b71cc00 con 0x55706b538000 2024-09-13T23:43:30.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.703+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 93941325084160] v33'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b675440 con 0x55706b538000 2024-09-13T23:43:30.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.703+0000 7f96f5e66f40 20 remove_watcher() i=6 2024-09-13T23:43:30.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.703+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93941325085312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55706b71d000 con 0x55706b538400 2024-09-13T23:43:30.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f4e20700 1 -- 172.21.15.167:0/2494398012 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 93941325085312] v33'95 uv93 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55706b674b40 con 0x55706b538400 2024-09-13T23:43:30.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 20 remove_watcher() i=7 2024-09-13T23:43:30.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55706b5cf400 msgr2=0x55706b504100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55706b5cf400 0x55706b504100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55706b5cf000 msgr2=0x55706b503b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55706b5cf000 0x55706b503b80 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55706b538400 msgr2=0x55706b504c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55706b538400 0x55706b504c00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55706b538000 msgr2=0x55706b504680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55706b538000 0x55706b504680 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b5cec00 msgr2=0x55706b503600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b5cec00 0x55706b503600 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x55706b4fd920 tx=0x55706b321800 comp rx=0 tx=0).stop 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 msgr2=0x55706b502b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/2494398012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d800 0x55706b502b00 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55706b53c330 tx=0x55706b56e000 comp rx=0 tx=0).stop 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 reap_dead start 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f3e1e700 1 -- 172.21.15.167:0/2494398012 reap_dead start 2024-09-13T23:43:30.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 shutdown_connections 2024-09-13T23:43:30.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 >> 172.21.15.167:0/2494398012 conn(0x55706b21d400 msgr2=0x55706b31d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 shutdown_connections 2024-09-13T23:43:30.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/2494398012 wait complete. 2024-09-13T23:43:30.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/921374824 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b71c000 msgr2=0x55706b722000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/921374824 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b71c000 0x55706b722000 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x55706b53d200 tx=0x55706b56f740 comp rx=0 tx=0).stop 2024-09-13T23:43:30.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/921374824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 msgr2=0x55706b505180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/921374824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 0x55706b505180 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x55706b69a1b0 tx=0x55706b69c300 comp rx=0 tx=0).stop 2024-09-13T23:43:30.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/921374824 shutdown_connections 2024-09-13T23:43:30.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/921374824 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b71c000 0x55706b722000 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/921374824 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b53ac00 0x55706b505180 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/921374824 >> 172.21.15.167:0/921374824 conn(0x55706b53bc00 msgr2=0x55706b31e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/921374824 shutdown_connections 2024-09-13T23:43:30.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/921374824 wait complete. 2024-09-13T23:43:30.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/3913665964 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b5ce000 msgr2=0x55706b503080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/3913665964 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b5ce000 0x55706b503080 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x55706b53d080 tx=0x55706b56eea0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/3913665964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 msgr2=0x55706b502580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/3913665964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 0x55706b502580 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55706bfe7e00 tx=0x55706b321920 comp rx=0 tx=0).stop 2024-09-13T23:43:30.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/3913665964 shutdown_connections 2024-09-13T23:43:30.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/3913665964 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55706b5ce000 0x55706b503080 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 --2- 172.21.15.167:0/3913665964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55706b21d000 0x55706b502580 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.707+0000 7f96f5e66f40 1 -- 172.21.15.167:0/3913665964 >> 172.21.15.167:0/3913665964 conn(0x55706b21cc00 msgr2=0x55706b31d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.711+0000 7f96f5e66f40 1 -- 172.21.15.167:0/3913665964 shutdown_connections 2024-09-13T23:43:30.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.711+0000 7f96f5e66f40 1 -- 172.21.15.167:0/3913665964 wait complete. 2024-09-13T23:43:30.721 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:30.721 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo", 2024-09-13T23:43:30.721 INFO:teuthology.orchestra.run.smithi167.stdout: "num_shards": 11, 2024-09-13T23:43:30.721 INFO:teuthology.orchestra.run.smithi167.stdout: "tenant": "", 2024-09-13T23:43:30.721 INFO:teuthology.orchestra.run.smithi167.stdout: "zonegroup": "363ea8c8-5049-424d-855a-cd6979bf7acf", 2024-09-13T23:43:30.721 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_rule": "default-placement", 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: "explicit_placement": { 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": "", 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "", 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": "" 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1", 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: "marker": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1", 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: "index_type": "Normal", 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: "versioned": false, 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: "versioning_enabled": false, 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: "object_lock_enabled": false, 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_enabled": false, 2024-09-13T23:43:30.722 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1", 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: "mtime": "2024-09-13T23:43:29.857870Z", 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: "creation_time": "2024-09-13T23:43:29.849424Z", 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: "usage": {}, 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:30.723 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:30.724 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:30.724 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '363ea8c8-5049-424d-855a-cd6979bf7acf', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1', 'marker': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-13T23:43:29.857870Z', 'creation_time': '2024-09-13T23:43:29.849424Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-09-13T23:43:30.725 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-09-13T23:43:30.725 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-09-13T23:43:30.725 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-09-13T23:43:30.762 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:30.763 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:30.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a768bef40 1 Processor -- start 2024-09-13T23:43:30.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a768bef40 1 -- start start 2024-09-13T23:43:30.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a768bef40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x564651805400 0x564651aea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a768bef40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5646518c9500 con 0x564651805000 2024-09-13T23:43:30.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a768bef40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x564651892a00 con 0x564651805400 2024-09-13T23:43:30.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a74876700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x564651805400 0x564651aea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a75077700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x564651805000 0x564651904800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45426/0 (socket says 172.21.15.167:45426) 2024-09-13T23:43:30.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a74876700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x564651805400 0x564651aea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51874/0 (socket says 172.21.15.167:51874) 2024-09-13T23:43:30.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a74876700 1 -- 172.21.15.167:0/2771547712 learned_addr learned my addr 172.21.15.167:0/2771547712 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a74876700 1 -- 172.21.15.167:0/2771547712 >> v1:172.21.15.12:6789/0 conn(0x564651805000 legacy=0x564651904800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a74075700 1 -- 172.21.15.167:0/2771547712 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351745831 0 0) 0x564651960000 con 0x564651805000 2024-09-13T23:43:30.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a74876700 1 -- 172.21.15.167:0/2771547712 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564651817d40 con 0x564651805400 2024-09-13T23:43:30.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a74876700 1 --2- 172.21.15.167:0/2771547712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805400 0x564651aea580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x564651ae5920 tx=0x564651909920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4ad638e341c93c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a74075700 1 -- 172.21.15.167:0/2771547712 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646518a94a0 con 0x564651805400 2024-09-13T23:43:30.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a74075700 1 -- 172.21.15.167:0/2771547712 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5646518c9800 con 0x564651805400 2024-09-13T23:43:30.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a74075700 1 -- 172.21.15.167:0/2771547712 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646518a9600 con 0x564651805400 2024-09-13T23:43:30.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a768bef40 1 -- 172.21.15.167:0/2771547712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805400 msgr2=0x564651aea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.755+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2771547712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805400 0x564651aea580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x564651ae5920 tx=0x564651909920 comp rx=0 tx=0).stop 2024-09-13T23:43:30.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 -- 172.21.15.167:0/2771547712 shutdown_connections 2024-09-13T23:43:30.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2771547712 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805400 0x564651aea580 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 -- 172.21.15.167:0/2771547712 >> 172.21.15.167:0/2771547712 conn(0x564651804c00 msgr2=0x564651904000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 -- 172.21.15.167:0/2771547712 shutdown_connections 2024-09-13T23:43:30.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 -- 172.21.15.167:0/2771547712 wait complete. 2024-09-13T23:43:30.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 Processor -- start 2024-09-13T23:43:30.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 -- start start 2024-09-13T23:43:30.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 0x564651aea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564651892500 con 0x564651805000 2024-09-13T23:43:30.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a75077700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 0x564651aea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a75077700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 0x564651aea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51884/0 (socket says 172.21.15.167:51884) 2024-09-13T23:43:30.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a75077700 1 -- 172.21.15.167:0/3469109315 learned_addr learned my addr 172.21.15.167:0/3469109315 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a75077700 1 -- 172.21.15.167:0/3469109315 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564651817ba0 con 0x564651805000 2024-09-13T23:43:30.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a75077700 1 --2- 172.21.15.167:0/3469109315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 0x564651aea580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5646525e3e30 tx=0x564651909b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4208e3f1a337c2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a73874700 1 -- 172.21.15.167:0/3469109315 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646518a91e0 con 0x564651805000 2024-09-13T23:43:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a73874700 1 -- 172.21.15.167:0/3469109315 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5646518c9b00 con 0x564651805000 2024-09-13T23:43:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a73874700 1 -- 172.21.15.167:0/3469109315 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646518a94a0 con 0x564651805000 2024-09-13T23:43:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 -- 172.21.15.167:0/3469109315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 msgr2=0x564651aea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 --2- 172.21.15.167:0/3469109315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 0x564651aea580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x5646525e3e30 tx=0x564651909b60 comp rx=0 tx=0).stop 2024-09-13T23:43:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 -- 172.21.15.167:0/3469109315 shutdown_connections 2024-09-13T23:43:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 --2- 172.21.15.167:0/3469109315 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 0x564651aea580 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 -- 172.21.15.167:0/3469109315 >> 172.21.15.167:0/3469109315 conn(0x564651804c00 msgr2=0x564651905000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.759+0000 7f5a768bef40 1 -- 172.21.15.167:0/3469109315 shutdown_connections 2024-09-13T23:43:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a768bef40 1 -- 172.21.15.167:0/3469109315 wait complete. 2024-09-13T23:43:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a768bef40 1 Processor -- start 2024-09-13T23:43:30.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a768bef40 1 -- start start 2024-09-13T23:43:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a768bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 0x564651aea580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a768bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646518923c0 con 0x564651805000 2024-09-13T23:43:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a75077700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 0x564651aea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a75077700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 0x564651aea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51890/0 (socket says 172.21.15.167:51890) 2024-09-13T23:43:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a75077700 1 -- 172.21.15.167:0/2912602412 learned_addr learned my addr 172.21.15.167:0/2912602412 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a75077700 1 -- 172.21.15.167:0/2912602412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564651817a00 con 0x564651805000 2024-09-13T23:43:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a75077700 1 --2- 172.21.15.167:0/2912602412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 0x564651aea580 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5646525e3f50 tx=0x564651909a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a72071700 1 -- 172.21.15.167:0/2912602412 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646518a9600 con 0x564651805000 2024-09-13T23:43:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a72071700 1 -- 172.21.15.167:0/2912602412 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5646518c9c80 con 0x564651805000 2024-09-13T23:43:30.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a72071700 1 -- 172.21.15.167:0/2912602412 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646518a91e0 con 0x564651805000 2024-09-13T23:43:30.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a768bef40 1 -- 172.21.15.167:0/2912602412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564651817860 con 0x564651805000 2024-09-13T23:43:30.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.763+0000 7f5a768bef40 1 -- 172.21.15.167:0/2912602412 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646518176c0 con 0x564651805000 2024-09-13T23:43:30.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a768bef40 1 Processor -- start 2024-09-13T23:43:30.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a768bef40 1 -- start start 2024-09-13T23:43:30.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a768bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 0x564651aeab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a768bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564651892000 con 0x564651805800 2024-09-13T23:43:30.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a72071700 1 -- 172.21.15.167:0/2912602412 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5646518feb00 con 0x564651805000 2024-09-13T23:43:30.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a72071700 1 --2- 172.21.15.167:0/2912602412 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651bba000 0x564651aeb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a72071700 1 -- 172.21.15.167:0/2912602412 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56465186b180 con 0x564651805000 2024-09-13T23:43:30.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a75077700 1 --2- 172.21.15.167:0/2912602412 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651bba000 0x564651aeb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a75878700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 0x564651aeab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a75878700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 0x564651aeab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51906/0 (socket says 172.21.15.167:51906) 2024-09-13T23:43:30.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a75878700 1 -- 172.21.15.167:0/3460049446 learned_addr learned my addr 172.21.15.167:0/3460049446 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a75878700 1 -- 172.21.15.167:0/3460049446 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564651816000 con 0x564651805800 2024-09-13T23:43:30.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a75077700 1 --2- 172.21.15.167:0/2912602412 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651bba000 0x564651aeb080 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x564651b29590 tx=0x564651b72240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a75878700 1 --2- 172.21.15.167:0/3460049446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 0x564651aeab00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x564651b28300 tx=0x564651b730e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181eac68ab17ebbf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a6f86c700 1 -- 172.21.15.167:0/3460049446 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646518a9600 con 0x564651805800 2024-09-13T23:43:30.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a6f86c700 1 -- 172.21.15.167:0/3460049446 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564651bcc000 con 0x564651805800 2024-09-13T23:43:30.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a6f86c700 1 -- 172.21.15.167:0/3460049446 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646518a91e0 con 0x564651805800 2024-09-13T23:43:30.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a768bef40 1 -- 172.21.15.167:0/3460049446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 msgr2=0x564651aeab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a768bef40 1 --2- 172.21.15.167:0/3460049446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 0x564651aeab00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x564651b28300 tx=0x564651b730e0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a768bef40 1 -- 172.21.15.167:0/3460049446 shutdown_connections 2024-09-13T23:43:30.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a768bef40 1 --2- 172.21.15.167:0/3460049446 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 0x564651aeab00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a768bef40 1 -- 172.21.15.167:0/3460049446 >> 172.21.15.167:0/3460049446 conn(0x564651805400 msgr2=0x564651905800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.767+0000 7f5a768bef40 1 -- 172.21.15.167:0/3460049446 shutdown_connections 2024-09-13T23:43:30.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a768bef40 1 -- 172.21.15.167:0/3460049446 wait complete. 2024-09-13T23:43:30.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a768bef40 1 Processor -- start 2024-09-13T23:43:30.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a768bef40 1 -- start start 2024-09-13T23:43:30.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a768bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 0x564651aeab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a768bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56465182a640 con 0x564651805800 2024-09-13T23:43:30.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a75878700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 0x564651aeab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a75878700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 0x564651aeab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51912/0 (socket says 172.21.15.167:51912) 2024-09-13T23:43:30.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 learned_addr learned my addr 172.21.15.167:0/2028009645 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564651816ea0 con 0x564651805800 2024-09-13T23:43:30.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a75878700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 0x564651aeab00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x564651b28390 tx=0x564651b72cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a6e069700 1 -- 172.21.15.167:0/2028009645 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646518a9a20 con 0x564651805800 2024-09-13T23:43:30.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a6e069700 1 -- 172.21.15.167:0/2028009645 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564651bcc180 con 0x564651805800 2024-09-13T23:43:30.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564651816820 con 0x564651805800 2024-09-13T23:43:30.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a6e069700 1 -- 172.21.15.167:0/2028009645 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646518a9600 con 0x564651805800 2024-09-13T23:43:30.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.771+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646518169c0 con 0x564651805800 2024-09-13T23:43:30.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a6e069700 1 -- 172.21.15.167:0/2028009645 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5646518ff080 con 0x564651805800 2024-09-13T23:43:30.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a6e069700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651bbac00 0x564651aeb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a6e069700 1 -- 172.21.15.167:0/2028009645 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56465186b340 con 0x564651805800 2024-09-13T23:43:30.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a74876700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651bbac00 0x564651aeb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564651bbb000 0x564651aebb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb400 con 0x564651bbb000 2024-09-13T23:43:30.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a75878700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564651bbb000 0x564651aebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a74876700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651bbac00 0x564651aeb600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x564651ae5920 tx=0x564651909800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a75878700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564651bbb000 0x564651aebb80 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 20 realm 2024-09-13T23:43:30.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb000 2024-09-13T23:43:30.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:30.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbbc00 con 0x564651bbb000 2024-09-13T23:43:30.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564651bbb400 0x564651aec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb400 2024-09-13T23:43:30.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.775+0000 7f5a75077700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564651bbb400 0x564651aec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a75077700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564651bbb400 0x564651aec100 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a768bef40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:30.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbbc00 con 0x564651bbb400 2024-09-13T23:43:30.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a768bef40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:30.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a768bef40 20 searching for the correct realm 2024-09-13T23:43:30.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564651b4a000 0x564651aec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4a400 con 0x564651b4a000 2024-09-13T23:43:30.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a74876700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564651b4a000 0x564651aec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a74876700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564651b4a000 0x564651aec680 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.779+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4a400 con 0x564651bbb400 2024-09-13T23:43:30.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a75077700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564651b4a400 0x564651aecc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a400 2024-09-13T23:43:30.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a75878700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564651b4a400 0x564651aecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a75878700 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564651b4a400 0x564651aecc00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb000 2024-09-13T23:43:30.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb000 2024-09-13T23:43:30.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a000 2024-09-13T23:43:30.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.783+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb000 2024-09-13T23:43:30.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.787+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.787+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb400 2024-09-13T23:43:30.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.787+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.787+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb400 2024-09-13T23:43:30.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.787+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.787+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb000 2024-09-13T23:43:30.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.787+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.787+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a400 2024-09-13T23:43:30.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.787+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.787+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb000 2024-09-13T23:43:30.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.787+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.787+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a400 2024-09-13T23:43:30.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb400 2024-09-13T23:43:30.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a000 2024-09-13T23:43:30.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb000 2024-09-13T23:43:30.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb400 2024-09-13T23:43:30.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a000 2024-09-13T23:43:30.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb000 2024-09-13T23:43:30.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.791+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb000 2024-09-13T23:43:30.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb000 2024-09-13T23:43:30.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a000 2024-09-13T23:43:30.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a400 2024-09-13T23:43:30.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb000 2024-09-13T23:43:30.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a400 2024-09-13T23:43:30.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a000 2024-09-13T23:43:30.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.795+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a400 2024-09-13T23:43:30.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb400 2024-09-13T23:43:30.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb400 2024-09-13T23:43:30.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a000 2024-09-13T23:43:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb400 2024-09-13T23:43:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb400 2024-09-13T23:43:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a768bef40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a768bef40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a768bef40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:30.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a768bef40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:30.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.799+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4a800 con 0x564651bbb000 2024-09-13T23:43:30.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4ac00 con 0x564651bbb400 2024-09-13T23:43:30.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:30.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4b000 con 0x564651bbb400 2024-09-13T23:43:30.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:30.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 20 zone default found 2024-09-13T23:43:30.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 4 Realm: () 2024-09-13T23:43:30.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 20 zonegroup default 2024-09-13T23:43:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4b400 con 0x564651b4a400 2024-09-13T23:43:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4b800 con 0x564651b4a000 2024-09-13T23:43:30.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4bc00 con 0x564651b4a000 2024-09-13T23:43:30.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.803+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.807+0000 7f5a768bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.807+0000 7f5a768bef40 20 started sync module instance, tier type = 2024-09-13T23:43:30.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.807+0000 7f5a768bef40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:30.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.807+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4b800 con 0x564651bbb400 2024-09-13T23:43:30.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.807+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.807+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94860018064128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4bc00 con 0x564651bbb400 2024-09-13T23:43:30.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.807+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4ac00 con 0x564651bbb000 2024-09-13T23:43:30.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.811+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.811+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94860018064128] v33'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.811+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94860018065280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4b000 con 0x564651bbb000 2024-09-13T23:43:30.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.811+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4b800 con 0x564651bbb400 2024-09-13T23:43:30.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.815+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94860018065280] v33'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.815+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.815+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94860018066432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbbc00 con 0x564651bbb400 2024-09-13T23:43:30.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.815+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651b4a400 2024-09-13T23:43:30.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.815+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94860018066432] v33'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.819+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.819+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94860018067584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4a800 con 0x564651b4a400 2024-09-13T23:43:30.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.819+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4b400 con 0x564651bbb400 2024-09-13T23:43:30.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.819+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94860018067584] v33'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.819+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.819+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94860018068736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4ac00 con 0x564651bbb400 2024-09-13T23:43:30.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.819+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4b000 con 0x564651b4a000 2024-09-13T23:43:30.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.823+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.823+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94860018069888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651c7e000 con 0x564651b4a000 2024-09-13T23:43:30.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.823+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651c7e400 con 0x564651b4a000 2024-09-13T23:43:30.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.823+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94860018068736] v33'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.823+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94860018069888] v33'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.823+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.823+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94860018071040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651c7e800 con 0x564651b4a000 2024-09-13T23:43:30.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.823+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651c7ec00 con 0x564651b4a400 2024-09-13T23:43:30.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'96 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94860018071040] v33'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94860018072192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651c7f000 con 0x564651b4a400 2024-09-13T23:43:30.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a768bef40 20 add_watcher() i=0 2024-09-13T23:43:30.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a768bef40 20 add_watcher() i=1 2024-09-13T23:43:30.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a768bef40 20 add_watcher() i=2 2024-09-13T23:43:30.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a768bef40 20 add_watcher() i=3 2024-09-13T23:43:30.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a768bef40 20 add_watcher() i=4 2024-09-13T23:43:30.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a768bef40 20 add_watcher() i=5 2024-09-13T23:43:30.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a768bef40 20 add_watcher() i=6 2024-09-13T23:43:30.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94860018072192] v33'97 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a768bef40 20 add_watcher() i=7 2024-09-13T23:43:30.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a768bef40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:30.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651c7f400 con 0x564651bbb000 2024-09-13T23:43:30.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.827+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94860019777536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651c7f800 con 0x564651bbb000 2024-09-13T23:43:30.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94860019777536] v33'98 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a5c045700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:30.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a768bef40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:30.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a768bef40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:30.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a768bef40 1 Processor -- start 2024-09-13T23:43:30.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a768bef40 1 -- start start 2024-09-13T23:43:30.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a768bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 0x564651aed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a768bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56465182a780 con 0x564651c7ec00 2024-09-13T23:43:30.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a74876700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 0x564651aed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a74876700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 0x564651aed180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51920/0 (socket says 172.21.15.167:51920) 2024-09-13T23:43:30.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a74876700 1 -- 172.21.15.167:0/4108568757 learned_addr learned my addr 172.21.15.167:0/4108568757 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a74876700 1 -- 172.21.15.167:0/4108568757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564651816b60 con 0x564651c7ec00 2024-09-13T23:43:30.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a74876700 1 --2- 172.21.15.167:0/4108568757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 0x564651aed180 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x564651c8c0c0 tx=0x564651909860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b8d187dca9883 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.831+0000 7f5a5b844700 1 -- 172.21.15.167:0/4108568757 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5646518a9760 con 0x564651c7ec00 2024-09-13T23:43:30.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a5b844700 1 -- 172.21.15.167:0/4108568757 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564651bcc600 con 0x564651c7ec00 2024-09-13T23:43:30.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a5b844700 1 -- 172.21.15.167:0/4108568757 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564651c92000 con 0x564651c7ec00 2024-09-13T23:43:30.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 -- 172.21.15.167:0/4108568757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 msgr2=0x564651aed180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 --2- 172.21.15.167:0/4108568757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 0x564651aed180 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x564651c8c0c0 tx=0x564651909860 comp rx=0 tx=0).stop 2024-09-13T23:43:30.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 -- 172.21.15.167:0/4108568757 shutdown_connections 2024-09-13T23:43:30.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 --2- 172.21.15.167:0/4108568757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 0x564651aed180 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 -- 172.21.15.167:0/4108568757 >> 172.21.15.167:0/4108568757 conn(0x564651c7fc00 msgr2=0x564651906000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 -- 172.21.15.167:0/4108568757 shutdown_connections 2024-09-13T23:43:30.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 -- 172.21.15.167:0/4108568757 wait complete. 2024-09-13T23:43:30.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 Processor -- start 2024-09-13T23:43:30.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 -- start start 2024-09-13T23:43:30.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 0x564651aed180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564651892000 con 0x564651c7ec00 2024-09-13T23:43:30.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a74876700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 0x564651aed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a74876700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 0x564651aed180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51934/0 (socket says 172.21.15.167:51934) 2024-09-13T23:43:30.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a74876700 1 -- 172.21.15.167:0/700441121 learned_addr learned my addr 172.21.15.167:0/700441121 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a74876700 1 -- 172.21.15.167:0/700441121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564651816d00 con 0x564651c7ec00 2024-09-13T23:43:30.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a74876700 1 --2- 172.21.15.167:0/700441121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 0x564651aed180 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x564651c8c300 tx=0x564651c8e300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a5a041700 1 -- 172.21.15.167:0/700441121 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564651c92160 con 0x564651c7ec00 2024-09-13T23:43:30.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a5a041700 1 -- 172.21.15.167:0/700441121 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564651bcc300 con 0x564651c7ec00 2024-09-13T23:43:30.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 -- 172.21.15.167:0/700441121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564651816340 con 0x564651c7ec00 2024-09-13T23:43:30.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a5a041700 1 -- 172.21.15.167:0/700441121 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564651c922c0 con 0x564651c7ec00 2024-09-13T23:43:30.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.835+0000 7f5a768bef40 1 -- 172.21.15.167:0/700441121 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564651817d40 con 0x564651c7ec00 2024-09-13T23:43:30.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.839+0000 7f5a5a041700 1 -- 172.21.15.167:0/700441121 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5646518ffb80 con 0x564651c7ec00 2024-09-13T23:43:30.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.839+0000 7f5a5a041700 1 --2- 172.21.15.167:0/700441121 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651d0c000 0x564651d12000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.839+0000 7f5a5a041700 1 -- 172.21.15.167:0/700441121 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56465186b500 con 0x564651c7ec00 2024-09-13T23:43:30.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.839+0000 7f5a75878700 1 --2- 172.21.15.167:0/700441121 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651d0c000 0x564651d12000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.839+0000 7f5a768bef40 5 note: GC not initialized 2024-09-13T23:43:30.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.839+0000 7f5a75878700 1 --2- 172.21.15.167:0/700441121 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651d0c000 0x564651d12000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x564651c137a0 tx=0x564651b737a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.839+0000 7f5a5783c700 20 reqs_thread_entry: start 2024-09-13T23:43:30.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.839+0000 7f5a768bef40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:30.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.839+0000 7f5a768bef40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:30.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.839+0000 7f5a55838700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:01 2024 2024-09-13T23:43:30.851 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:30.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a55016700 20 reqs_thread_entry: start 2024-09-13T23:43:30.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651c7e000 con 0x564651bbb000 2024-09-13T23:43:30.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:30.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651c7e400 con 0x564651b4a400 2024-09-13T23:43:30.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4a800 con 0x564651b4a400 2024-09-13T23:43:30.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:30.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4b000 con 0x564651b4a000 2024-09-13T23:43:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(65 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-13T23:43:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 4.a 4:517e2946:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb400 2024-09-13T23:43:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 4.8 4:10b47b19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651d0c400 con 0x564651b4a000 2024-09-13T23:43:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 4.2e 4:75bb53d2:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651d0c800 con 0x564651b4a000 2024-09-13T23:43:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 4.26 4:6424e2c9:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651d0cc00 con 0x564651bbb000 2024-09-13T23:43:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 4.c 4:32a29db1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651d0d000 con 0x564651bbb000 2024-09-13T23:43:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 4.28 4:162fa60a:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651d0d400 con 0x564651bbb400 2024-09-13T23:43:30.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 4.2c 4:355fcb42:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651d0d800 con 0x564651bbb000 2024-09-13T23:43:30.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 4.2f 4:f64e09ba:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651d0dc00 con 0x564651bbb000 2024-09-13T23:43:30.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.843+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:74 4.3a 4:5ee337b5:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbbc00 con 0x564651b4a000 2024-09-13T23:43:30.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:75 4.2c 4:3559f32d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651bbb800 con 0x564651bbb000 2024-09-13T23:43:30.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 4.1 4:8131d739:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4a800 con 0x564651b4a400 2024-09-13T23:43:30.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(66 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(69 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(71 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.5 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(70 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(73 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(72 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(67 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(68 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(74 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(76 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(75 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.9 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a5c045700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:30.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.847+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94860019777536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4bc00 con 0x564651bbb000 2024-09-13T23:43:30.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.851+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94860019777536] v33'99 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.851+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94860018064128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4b800 con 0x564651bbb400 2024-09-13T23:43:30.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.851+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94860018064128] v33'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.851+0000 7f5a768bef40 20 remove_watcher() i=0 2024-09-13T23:43:30.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.851+0000 7f5a768bef40 2 removed watcher, disabling cache 2024-09-13T23:43:30.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.851+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94860018065280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4b400 con 0x564651bbb000 2024-09-13T23:43:30.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.855+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94860018065280] v33'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651bbb000 2024-09-13T23:43:30.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.855+0000 7f5a768bef40 20 remove_watcher() i=1 2024-09-13T23:43:30.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.855+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94860018066432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651b4ac00 con 0x564651bbb400 2024-09-13T23:43:30.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.855+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94860018066432] v33'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.855+0000 7f5a768bef40 20 remove_watcher() i=2 2024-09-13T23:43:30.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.855+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94860018067584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651d0cc00 con 0x564651b4a400 2024-09-13T23:43:30.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.859+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94860018067584] v33'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.859+0000 7f5a768bef40 20 remove_watcher() i=3 2024-09-13T23:43:30.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.859+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94860018068736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651d0d000 con 0x564651bbb400 2024-09-13T23:43:30.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.863+0000 7f5a75077700 1 -- 172.21.15.167:0/2028009645 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94860018068736] v33'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65200 con 0x564651bbb400 2024-09-13T23:43:30.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.863+0000 7f5a768bef40 20 remove_watcher() i=4 2024-09-13T23:43:30.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.863+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94860018069888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651c7f800 con 0x564651b4a000 2024-09-13T23:43:30.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.863+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94860018069888] v33'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.863+0000 7f5a768bef40 20 remove_watcher() i=5 2024-09-13T23:43:30.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.863+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94860018071040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651c7e000 con 0x564651b4a000 2024-09-13T23:43:30.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94860018071040] v33'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c65440 con 0x564651b4a000 2024-09-13T23:43:30.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 20 remove_watcher() i=6 2024-09-13T23:43:30.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94860018072192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564651c7e400 con 0x564651b4a400 2024-09-13T23:43:30.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a75878700 1 -- 172.21.15.167:0/2028009645 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94860018072192] v33'98 uv96 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564651c64b40 con 0x564651b4a400 2024-09-13T23:43:30.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 20 remove_watcher() i=7 2024-09-13T23:43:30.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564651bbb400 msgr2=0x564651aec100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564651bbb400 0x564651aec100 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564651bbb000 msgr2=0x564651aebb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564651bbb000 0x564651aebb80 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564651b4a400 msgr2=0x564651aecc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564651b4a400 0x564651aecc00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564651b4a000 msgr2=0x564651aec680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564651b4a000 0x564651aec680 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651bbac00 msgr2=0x564651aeb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651bbac00 0x564651aeb600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x564651ae5920 tx=0x564651909800 comp rx=0 tx=0).stop 2024-09-13T23:43:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 msgr2=0x564651aeab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2028009645 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805800 0x564651aeab00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x564651b28390 tx=0x564651b72cc0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 reap_dead start 2024-09-13T23:43:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a74876700 1 -- 172.21.15.167:0/2028009645 reap_dead start 2024-09-13T23:43:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 shutdown_connections 2024-09-13T23:43:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 >> 172.21.15.167:0/2028009645 conn(0x564651805400 msgr2=0x564651905800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 shutdown_connections 2024-09-13T23:43:30.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.867+0000 7f5a768bef40 1 -- 172.21.15.167:0/2028009645 wait complete. 2024-09-13T23:43:30.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 -- 172.21.15.167:0/700441121 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651d0c000 msgr2=0x564651d12000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 --2- 172.21.15.167:0/700441121 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651d0c000 0x564651d12000 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x564651c137a0 tx=0x564651b737a0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 -- 172.21.15.167:0/700441121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 msgr2=0x564651aed180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 --2- 172.21.15.167:0/700441121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 0x564651aed180 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x564651c8c300 tx=0x564651c8e300 comp rx=0 tx=0).stop 2024-09-13T23:43:30.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 -- 172.21.15.167:0/700441121 shutdown_connections 2024-09-13T23:43:30.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 --2- 172.21.15.167:0/700441121 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651d0c000 0x564651d12000 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 --2- 172.21.15.167:0/700441121 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651c7ec00 0x564651aed180 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 -- 172.21.15.167:0/700441121 >> 172.21.15.167:0/700441121 conn(0x564651c7fc00 msgr2=0x564651906000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 -- 172.21.15.167:0/700441121 shutdown_connections 2024-09-13T23:43:30.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 -- 172.21.15.167:0/700441121 wait complete. 2024-09-13T23:43:30.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 -- 172.21.15.167:0/2912602412 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651bba000 msgr2=0x564651aeb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2912602412 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651bba000 0x564651aeb080 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x564651b29590 tx=0x564651b72240 comp rx=0 tx=0).stop 2024-09-13T23:43:30.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 -- 172.21.15.167:0/2912602412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 msgr2=0x564651aea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2912602412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 0x564651aea580 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x5646525e3f50 tx=0x564651909a40 comp rx=0 tx=0).stop 2024-09-13T23:43:30.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 -- 172.21.15.167:0/2912602412 shutdown_connections 2024-09-13T23:43:30.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2912602412 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564651bba000 0x564651aeb080 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 --2- 172.21.15.167:0/2912602412 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564651805000 0x564651aea580 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 -- 172.21.15.167:0/2912602412 >> 172.21.15.167:0/2912602412 conn(0x564651804c00 msgr2=0x564651905000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 -- 172.21.15.167:0/2912602412 shutdown_connections 2024-09-13T23:43:30.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.871+0000 7f5a768bef40 1 -- 172.21.15.167:0/2912602412 wait complete. 2024-09-13T23:43:30.883 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-09-13T23:43:30.884 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:30.884 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo", 2024-09-13T23:43:30.884 INFO:teuthology.orchestra.run.smithi167.stdout: "num_shards": 11, 2024-09-13T23:43:30.884 INFO:teuthology.orchestra.run.smithi167.stdout: "tenant": "", 2024-09-13T23:43:30.884 INFO:teuthology.orchestra.run.smithi167.stdout: "zonegroup": "363ea8c8-5049-424d-855a-cd6979bf7acf", 2024-09-13T23:43:30.884 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_rule": "default-placement", 2024-09-13T23:43:30.884 INFO:teuthology.orchestra.run.smithi167.stdout: "explicit_placement": { 2024-09-13T23:43:30.884 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": "", 2024-09-13T23:43:30.884 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "", 2024-09-13T23:43:30.884 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": "" 2024-09-13T23:43:30.884 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:30.884 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1", 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "marker": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1", 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "index_type": "Normal", 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "versioned": false, 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "versioning_enabled": false, 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "object_lock_enabled": false, 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_enabled": false, 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1", 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "mtime": "2024-09-13T23:43:29.857870Z", 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "creation_time": "2024-09-13T23:43:29.849424Z", 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2024-09-13T23:43:30.885 INFO:teuthology.orchestra.run.smithi167.stdout: "usage": {}, 2024-09-13T23:43:30.886 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:30.886 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:30.886 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:30.886 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:30.886 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:30.886 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:30.886 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:30.886 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:30.886 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-09-13T23:43:30.887 DEBUG:tasks.util.rgw: json result: [{'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '363ea8c8-5049-424d-855a-cd6979bf7acf', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1', 'marker': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-13T23:43:29.857870Z', 'creation_time': '2024-09-13T23:43:29.849424Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2024-09-13T23:43:30.900 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-09-13T23:43:30.900 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-09-13T23:43:30.901 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-09-13T23:43:30.901 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-09-13T23:43:30.973 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:30.973 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:30.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.963+0000 7ff4c4e99f40 1 Processor -- start 2024-09-13T23:43:30.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.963+0000 7ff4c4e99f40 1 -- start start 2024-09-13T23:43:30.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.963+0000 7ff4c4e99f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5643556d9400 0x5643559be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.963+0000 7ff4c4e99f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56435579d500 con 0x5643556d9000 2024-09-13T23:43:30.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.963+0000 7ff4c4e99f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x564355766a00 con 0x5643556d9400 2024-09-13T23:43:30.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.963+0000 7ff4c2e51700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5643556d9400 0x5643559be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.963+0000 7ff4c3652700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5643556d9000 0x5643557d8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45440/0 (socket says 172.21.15.167:45440) 2024-09-13T23:43:30.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.963+0000 7ff4c2e51700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5643556d9400 0x5643559be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51940/0 (socket says 172.21.15.167:51940) 2024-09-13T23:43:30.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.963+0000 7ff4c3652700 1 -- 172.21.15.167:0/398065003 learned_addr learned my addr 172.21.15.167:0/398065003 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:30.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c2650700 1 -- 172.21.15.167:0/398065003 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256607670 0 0) 0x564355834000 con 0x5643556d9000 2024-09-13T23:43:30.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c2650700 1 -- 172.21.15.167:0/398065003 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x56435579d680 con 0x5643556d9000 2024-09-13T23:43:30.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c2e51700 1 -- 172.21.15.167:0/398065003 >> v1:172.21.15.12:6789/0 conn(0x5643556d9000 legacy=0x5643557d8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c2e51700 1 -- 172.21.15.167:0/398065003 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643556ebd40 con 0x5643556d9400 2024-09-13T23:43:30.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c2e51700 1 --2- 172.21.15.167:0/398065003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9400 0x5643559be580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x5643559b9920 tx=0x5643557dd980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b8e7ca84126ab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c2650700 1 -- 172.21.15.167:0/398065003 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56435577d4a0 con 0x5643556d9400 2024-09-13T23:43:30.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c2650700 1 -- 172.21.15.167:0/398065003 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56435579d680 con 0x5643556d9400 2024-09-13T23:43:30.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c2650700 1 -- 172.21.15.167:0/398065003 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56435577d600 con 0x5643556d9400 2024-09-13T23:43:30.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/398065003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9400 msgr2=0x5643559be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/398065003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9400 0x5643559be580 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x5643559b9920 tx=0x5643557dd980 comp rx=0 tx=0).stop 2024-09-13T23:43:30.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/398065003 shutdown_connections 2024-09-13T23:43:30.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/398065003 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9400 0x5643559be580 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/398065003 >> 172.21.15.167:0/398065003 conn(0x5643556d8c00 msgr2=0x5643557d8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/398065003 shutdown_connections 2024-09-13T23:43:30.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/398065003 wait complete. 2024-09-13T23:43:30.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c4e99f40 1 Processor -- start 2024-09-13T23:43:30.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c4e99f40 1 -- start start 2024-09-13T23:43:30.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c4e99f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 0x5643559be580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c4e99f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564355766500 con 0x5643556d9000 2024-09-13T23:43:30.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c3652700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 0x5643559be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c3652700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 0x5643559be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51946/0 (socket says 172.21.15.167:51946) 2024-09-13T23:43:30.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.967+0000 7ff4c3652700 1 -- 172.21.15.167:0/2867258383 learned_addr learned my addr 172.21.15.167:0/2867258383 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c3652700 1 -- 172.21.15.167:0/2867258383 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643556ebba0 con 0x5643556d9000 2024-09-13T23:43:30.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c3652700 1 --2- 172.21.15.167:0/2867258383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 0x5643559be580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5643564a3d70 tx=0x5643557ddb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b26f35447c7f806 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c1e4f700 1 -- 172.21.15.167:0/2867258383 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56435577d4a0 con 0x5643556d9000 2024-09-13T23:43:30.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c1e4f700 1 -- 172.21.15.167:0/2867258383 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56435579db00 con 0x5643556d9000 2024-09-13T23:43:30.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c1e4f700 1 -- 172.21.15.167:0/2867258383 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56435577d600 con 0x5643556d9000 2024-09-13T23:43:30.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/2867258383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 msgr2=0x5643559be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/2867258383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 0x5643559be580 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5643564a3d70 tx=0x5643557ddb60 comp rx=0 tx=0).stop 2024-09-13T23:43:30.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/2867258383 shutdown_connections 2024-09-13T23:43:30.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/2867258383 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 0x5643559be580 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/2867258383 >> 172.21.15.167:0/2867258383 conn(0x5643556d8c00 msgr2=0x5643557d9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/2867258383 shutdown_connections 2024-09-13T23:43:30.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/2867258383 wait complete. 2024-09-13T23:43:30.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c4e99f40 1 Processor -- start 2024-09-13T23:43:30.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c4e99f40 1 -- start start 2024-09-13T23:43:30.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c4e99f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 0x5643559be580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c4e99f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643557663c0 con 0x5643556d9000 2024-09-13T23:43:30.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c3652700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 0x5643559be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c3652700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 0x5643559be580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51958/0 (socket says 172.21.15.167:51958) 2024-09-13T23:43:30.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c3652700 1 -- 172.21.15.167:0/1556175037 learned_addr learned my addr 172.21.15.167:0/1556175037 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.971+0000 7ff4c3652700 1 -- 172.21.15.167:0/1556175037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643556eba00 con 0x5643556d9000 2024-09-13T23:43:30.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c3652700 1 --2- 172.21.15.167:0/1556175037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 0x5643559be580 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x5643564a3e60 tx=0x5643557dd920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c064c700 1 -- 172.21.15.167:0/1556175037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56435577d8c0 con 0x5643556d9000 2024-09-13T23:43:30.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c064c700 1 -- 172.21.15.167:0/1556175037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56435579dc80 con 0x5643556d9000 2024-09-13T23:43:30.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1556175037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5643556eb860 con 0x5643556d9000 2024-09-13T23:43:30.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c064c700 1 -- 172.21.15.167:0/1556175037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56435577d4a0 con 0x5643556d9000 2024-09-13T23:43:30.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1556175037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643556eb6c0 con 0x5643556d9000 2024-09-13T23:43:30.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c4e99f40 1 Processor -- start 2024-09-13T23:43:30.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c4e99f40 1 -- start start 2024-09-13T23:43:30.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c064c700 1 -- 172.21.15.167:0/1556175037 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5643557d2b00 con 0x5643556d9000 2024-09-13T23:43:30.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c4e99f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 0x5643559beb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c4e99f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564355766000 con 0x5643556d9800 2024-09-13T23:43:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c064c700 1 --2- 172.21.15.167:0/1556175037 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355a80000 0x5643559bf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c3e53700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 0x5643559beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c064c700 1 -- 172.21.15.167:0/1556175037 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56435573f180 con 0x5643556d9000 2024-09-13T23:43:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c3e53700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 0x5643559beb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51968/0 (socket says 172.21.15.167:51968) 2024-09-13T23:43:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c3e53700 1 -- 172.21.15.167:0/1845768605 learned_addr learned my addr 172.21.15.167:0/1845768605 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c3652700 1 --2- 172.21.15.167:0/1556175037 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355a80000 0x5643559bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.975+0000 7ff4c3e53700 1 -- 172.21.15.167:0/1845768605 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643556ea000 con 0x5643556d9800 2024-09-13T23:43:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c3e53700 1 --2- 172.21.15.167:0/1845768605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 0x5643559beb00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x5643559b9f80 tx=0x564355a38de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d4229e290a27d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c3652700 1 --2- 172.21.15.167:0/1556175037 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355a80000 0x5643559bf080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x564355a054d0 tx=0x564355a38240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4bde47700 1 -- 172.21.15.167:0/1845768605 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56435577d8c0 con 0x5643556d9800 2024-09-13T23:43:30.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4bde47700 1 -- 172.21.15.167:0/1845768605 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564355a8c000 con 0x5643556d9800 2024-09-13T23:43:30.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4bde47700 1 -- 172.21.15.167:0/1845768605 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56435577d4a0 con 0x5643556d9800 2024-09-13T23:43:30.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1845768605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 msgr2=0x5643559beb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:30.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/1845768605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 0x5643559beb00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x5643559b9f80 tx=0x564355a38de0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1845768605 shutdown_connections 2024-09-13T23:43:30.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/1845768605 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 0x5643559beb00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:30.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1845768605 >> 172.21.15.167:0/1845768605 conn(0x5643556d9400 msgr2=0x5643557d9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:30.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1845768605 shutdown_connections 2024-09-13T23:43:30.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1845768605 wait complete. 2024-09-13T23:43:30.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c4e99f40 1 Processor -- start 2024-09-13T23:43:30.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c4e99f40 1 -- start start 2024-09-13T23:43:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c4e99f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 0x5643559beb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c4e99f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643556fe640 con 0x5643556d9800 2024-09-13T23:43:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c3e53700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 0x5643559beb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c3e53700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 0x5643559beb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51980/0 (socket says 172.21.15.167:51980) 2024-09-13T23:43:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 learned_addr learned my addr 172.21.15.167:0/4272765924 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643556eaea0 con 0x5643556d9800 2024-09-13T23:43:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.979+0000 7ff4c3e53700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 0x5643559beb00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x564355a04360 tx=0x5643557ddbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4bc644700 1 -- 172.21.15.167:0/4272765924 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56435577db80 con 0x5643556d9800 2024-09-13T23:43:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4bc644700 1 -- 172.21.15.167:0/4272765924 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564355a8c180 con 0x5643556d9800 2024-09-13T23:43:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4bc644700 1 -- 172.21.15.167:0/4272765924 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56435577d8c0 con 0x5643556d9800 2024-09-13T23:43:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5643556ea820 con 0x5643556d9800 2024-09-13T23:43:30.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643556ea9c0 con 0x5643556d9800 2024-09-13T23:43:30.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4bc644700 1 -- 172.21.15.167:0/4272765924 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5643557d2dc0 con 0x5643556d9800 2024-09-13T23:43:30.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4bc644700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355a80c00 0x5643559bf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4bc644700 1 -- 172.21.15.167:0/4272765924 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56435573f340 con 0x5643556d9800 2024-09-13T23:43:30.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4c2e51700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355a80c00 0x5643559bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564355a81000 0x5643559bfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81400 con 0x564355a81000 2024-09-13T23:43:30.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4c3e53700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564355a81000 0x5643559bfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4c2e51700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355a80c00 0x5643559bf600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5643559b9920 tx=0x5643557dd800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.983+0000 7ff4c3e53700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564355a81000 0x5643559bfb80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 20 realm 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81000 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81c00 con 0x564355a81000 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:30.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564355a81400 0x5643559c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81400 2024-09-13T23:43:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c3652700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564355a81400 0x5643559c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c3652700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564355a81400 0x5643559c0100 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:30.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:30.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:30.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:30.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81c00 con 0x564355a81400 2024-09-13T23:43:30.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:30.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:30.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 20 searching for the correct realm 2024-09-13T23:43:30.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564355a1a000 0x5643559c0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:30.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1a400 con 0x564355a1a000 2024-09-13T23:43:30.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.987+0000 7ff4c2e51700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564355a1a000 0x5643559c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:30.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c2e51700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564355a1a000 0x5643559c0680 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1a400 con 0x564355a81400 2024-09-13T23:43:31.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c3652700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564355a1a400 0x5643559c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a400 2024-09-13T23:43:31.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c3e53700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564355a1a400 0x5643559c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c3e53700 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564355a1a400 0x5643559c0c00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81000 2024-09-13T23:43:31.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81000 2024-09-13T23:43:31.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.991+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a000 2024-09-13T23:43:31.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81000 2024-09-13T23:43:31.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81400 2024-09-13T23:43:31.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81400 2024-09-13T23:43:31.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81000 2024-09-13T23:43:31.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a400 2024-09-13T23:43:31.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81000 2024-09-13T23:43:31.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.995+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a400 2024-09-13T23:43:31.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81400 2024-09-13T23:43:31.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a000 2024-09-13T23:43:31.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81000 2024-09-13T23:43:31.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81400 2024-09-13T23:43:31.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a000 2024-09-13T23:43:31.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81000 2024-09-13T23:43:31.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:30.999+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81000 2024-09-13T23:43:31.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81000 2024-09-13T23:43:31.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a000 2024-09-13T23:43:31.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a400 2024-09-13T23:43:31.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81000 2024-09-13T23:43:31.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a400 2024-09-13T23:43:31.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a000 2024-09-13T23:43:31.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.003+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a400 2024-09-13T23:43:31.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81400 2024-09-13T23:43:31.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81400 2024-09-13T23:43:31.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a000 2024-09-13T23:43:31.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81400 2024-09-13T23:43:31.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81400 2024-09-13T23:43:31.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c4e99f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:31.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c4e99f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:31.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c4e99f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:31.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c4e99f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:31.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.007+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1a800 con 0x564355a81000 2024-09-13T23:43:31.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1ac00 con 0x564355a81400 2024-09-13T23:43:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1b000 con 0x564355a81400 2024-09-13T23:43:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 zone default found 2024-09-13T23:43:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 4 Realm: () 2024-09-13T23:43:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 zonegroup default 2024-09-13T23:43:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1b400 con 0x564355a1a400 2024-09-13T23:43:31.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1b800 con 0x564355a1a000 2024-09-13T23:43:31.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1bc00 con 0x564355a1a000 2024-09-13T23:43:31.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 started sync module instance, tier type = 2024-09-13T23:43:31.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:31.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.011+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1b800 con 0x564355a81400 2024-09-13T23:43:31.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.015+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.015+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94847199042304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1bc00 con 0x564355a81400 2024-09-13T23:43:31.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.015+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1ac00 con 0x564355a81000 2024-09-13T23:43:31.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.019+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.019+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94847199043456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1b000 con 0x564355a81000 2024-09-13T23:43:31.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.019+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1b800 con 0x564355a81400 2024-09-13T23:43:31.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.019+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94847199042304] v33'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.019+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94847199043456] v33'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.023+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.023+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94847199044608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81c00 con 0x564355a81400 2024-09-13T23:43:31.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.023+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a1a400 2024-09-13T23:43:31.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.023+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94847199044608] v33'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.023+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.023+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94847199045760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1a800 con 0x564355a1a400 2024-09-13T23:43:31.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.023+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1b400 con 0x564355a81400 2024-09-13T23:43:31.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.027+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94847199045760] v33'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.027+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.027+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94847199046912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1ac00 con 0x564355a81400 2024-09-13T23:43:31.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.027+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1b000 con 0x564355a1a000 2024-09-13T23:43:31.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.027+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.031+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94847199048064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1c000 con 0x564355a1a000 2024-09-13T23:43:31.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.031+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1c400 con 0x564355a1a000 2024-09-13T23:43:31.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.031+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94847199046912] v33'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.031+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94847199048064] v33'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.031+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.031+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94847199049216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1c800 con 0x564355a1a000 2024-09-13T23:43:31.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.031+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1cc00 con 0x564355a1a400 2024-09-13T23:43:31.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94847199049216] v33'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'99 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94847199050368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1d000 con 0x564355a1a400 2024-09-13T23:43:31.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c4e99f40 20 add_watcher() i=0 2024-09-13T23:43:31.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c4e99f40 20 add_watcher() i=1 2024-09-13T23:43:31.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c4e99f40 20 add_watcher() i=2 2024-09-13T23:43:31.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c4e99f40 20 add_watcher() i=3 2024-09-13T23:43:31.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c4e99f40 20 add_watcher() i=4 2024-09-13T23:43:31.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c4e99f40 20 add_watcher() i=5 2024-09-13T23:43:31.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c4e99f40 20 add_watcher() i=6 2024-09-13T23:43:31.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94847199050368] v33'100 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c4e99f40 20 add_watcher() i=7 2024-09-13T23:43:31.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c4e99f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:31.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1d400 con 0x564355a81000 2024-09-13T23:43:31.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.035+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94847200722944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1d800 con 0x564355a81000 2024-09-13T23:43:31.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94847200722944] v33'100 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4aa620700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:31.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4c4e99f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:31.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4c4e99f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:31.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4c4e99f40 1 Processor -- start 2024-09-13T23:43:31.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4c4e99f40 1 -- start start 2024-09-13T23:43:31.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4c4e99f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 0x5643559c1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4c4e99f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643556fe780 con 0x564355a1cc00 2024-09-13T23:43:31.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4c2e51700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 0x5643559c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4c2e51700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 0x5643559c1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:51992/0 (socket says 172.21.15.167:51992) 2024-09-13T23:43:31.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4c2e51700 1 -- 172.21.15.167:0/3008974396 learned_addr learned my addr 172.21.15.167:0/3008974396 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4c2e51700 1 -- 172.21.15.167:0/3008974396 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643556eab60 con 0x564355a1cc00 2024-09-13T23:43:31.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4c2e51700 1 --2- 172.21.15.167:0/3008974396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 0x5643559c1180 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x564355a044b0 tx=0x5643557dd860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7324496c5a117d37 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4a9e1f700 1 -- 172.21.15.167:0/3008974396 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56435577d760 con 0x564355a1cc00 2024-09-13T23:43:31.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4a9e1f700 1 -- 172.21.15.167:0/3008974396 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564355a8c600 con 0x564355a1cc00 2024-09-13T23:43:31.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.039+0000 7ff4a9e1f700 1 -- 172.21.15.167:0/3008974396 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564355b64000 con 0x564355a1cc00 2024-09-13T23:43:31.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/3008974396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 msgr2=0x5643559c1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/3008974396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 0x5643559c1180 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x564355a044b0 tx=0x5643557dd860 comp rx=0 tx=0).stop 2024-09-13T23:43:31.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/3008974396 shutdown_connections 2024-09-13T23:43:31.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/3008974396 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 0x5643559c1180 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/3008974396 >> 172.21.15.167:0/3008974396 conn(0x564355a1dc00 msgr2=0x5643557da000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/3008974396 shutdown_connections 2024-09-13T23:43:31.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/3008974396 wait complete. 2024-09-13T23:43:31.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 Processor -- start 2024-09-13T23:43:31.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 -- start start 2024-09-13T23:43:31.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 0x5643559c1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564355766000 con 0x564355a1cc00 2024-09-13T23:43:31.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c2e51700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 0x5643559c1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c2e51700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 0x5643559c1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52006/0 (socket says 172.21.15.167:52006) 2024-09-13T23:43:31.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c2e51700 1 -- 172.21.15.167:0/928901722 learned_addr learned my addr 172.21.15.167:0/928901722 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c2e51700 1 -- 172.21.15.167:0/928901722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643556ead00 con 0x564355a1cc00 2024-09-13T23:43:31.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c2e51700 1 --2- 172.21.15.167:0/928901722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 0x5643559c1180 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x564355b587e0 tx=0x564355b5a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4a861c700 1 -- 172.21.15.167:0/928901722 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564355b64160 con 0x564355a1cc00 2024-09-13T23:43:31.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4a861c700 1 -- 172.21.15.167:0/928901722 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564355a8c300 con 0x564355a1cc00 2024-09-13T23:43:31.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4a861c700 1 -- 172.21.15.167:0/928901722 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564355b642c0 con 0x564355a1cc00 2024-09-13T23:43:31.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/928901722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5643556ea340 con 0x564355a1cc00 2024-09-13T23:43:31.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.043+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/928901722 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643556ebd40 con 0x564355a1cc00 2024-09-13T23:43:31.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4a861c700 1 -- 172.21.15.167:0/928901722 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5643557d3b80 con 0x564355a1cc00 2024-09-13T23:43:31.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4a861c700 1 --2- 172.21.15.167:0/928901722 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355bd8000 0x564355bde000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4a861c700 1 -- 172.21.15.167:0/928901722 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56435573f500 con 0x564355a1cc00 2024-09-13T23:43:31.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4c3e53700 1 --2- 172.21.15.167:0/928901722 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355bd8000 0x564355bde000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4c4e99f40 5 note: GC not initialized 2024-09-13T23:43:31.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4c3e53700 1 --2- 172.21.15.167:0/928901722 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355bd8000 0x564355bde000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x564355ad9d70 tx=0x564355a397a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4a5e17700 20 reqs_thread_entry: start 2024-09-13T23:43:31.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4c4e99f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:31.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4c4e99f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:31.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4a3e13700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:01 2024 2024-09-13T23:43:31.058 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:31.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4a35f1700 20 reqs_thread_entry: start 2024-09-13T23:43:31.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.047+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1c000 con 0x564355a1a400 2024-09-13T23:43:31.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:31.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1c400 con 0x564355a1a000 2024-09-13T23:43:31.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:31.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-13T23:43:31.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1a800 con 0x564355a1a400 2024-09-13T23:43:31.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:31.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1b000 con 0x564355a1a000 2024-09-13T23:43:31.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(65 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:31.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-13T23:43:31.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 4.a 4:517e2946:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81400 2024-09-13T23:43:31.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 4.8 4:10b47b19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355bd8400 con 0x564355a1a000 2024-09-13T23:43:31.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 4.2e 4:75bb53d2:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355bd8800 con 0x564355a1a000 2024-09-13T23:43:31.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 4.26 4:6424e2c9:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355bd8c00 con 0x564355a81000 2024-09-13T23:43:31.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 4.c 4:32a29db1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355bd9000 con 0x564355a81000 2024-09-13T23:43:31.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 4.28 4:162fa60a:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355bd9400 con 0x564355a81400 2024-09-13T23:43:31.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 4.2c 4:355fcb42:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355bd9800 con 0x564355a81000 2024-09-13T23:43:31.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 4.2f 4:f64e09ba:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355bd9c00 con 0x564355a81000 2024-09-13T23:43:31.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:74 4.3a 4:5ee337b5:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355bd8c00 con 0x564355a1a000 2024-09-13T23:43:31.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:75 4.2c 4:3559f32d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355bd9000 con 0x564355a81000 2024-09-13T23:43:31.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.051+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 4.1 4:8131d739:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355bd9800 con 0x564355a1a400 2024-09-13T23:43:31.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.055+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(68 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.055+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(67 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.055+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(74 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.055+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(71 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.5 [call out=124b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.055+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.055+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(69 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.055+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(76 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.055+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(70 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.055+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(72 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.055+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(73 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.055+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(75 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.9 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.059+0000 7ff4aa620700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:31.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.059+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94847200722944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1b000 con 0x564355a81000 2024-09-13T23:43:31.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.059+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94847200722944] v33'101 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.059+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94847199042304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1d800 con 0x564355a81400 2024-09-13T23:43:31.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.063+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94847199042304] v33'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.063+0000 7ff4c4e99f40 20 remove_watcher() i=0 2024-09-13T23:43:31.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.063+0000 7ff4c4e99f40 2 removed watcher, disabling cache 2024-09-13T23:43:31.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.063+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94847199043456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1c000 con 0x564355a81000 2024-09-13T23:43:31.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.063+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94847199043456] v33'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a81000 2024-09-13T23:43:31.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.063+0000 7ff4c4e99f40 20 remove_watcher() i=1 2024-09-13T23:43:31.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.063+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94847199044608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1c400 con 0x564355a81400 2024-09-13T23:43:31.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.067+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94847199044608] v33'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.067+0000 7ff4c4e99f40 20 remove_watcher() i=2 2024-09-13T23:43:31.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.067+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94847199045760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81c00 con 0x564355a1a400 2024-09-13T23:43:31.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.067+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94847199045760] v33'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.067+0000 7ff4c4e99f40 20 remove_watcher() i=3 2024-09-13T23:43:31.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.067+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94847199046912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a81800 con 0x564355a81400 2024-09-13T23:43:31.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.071+0000 7ff4c3652700 1 -- 172.21.15.167:0/4272765924 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94847199046912] v33'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37200 con 0x564355a81400 2024-09-13T23:43:31.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.071+0000 7ff4c4e99f40 20 remove_watcher() i=4 2024-09-13T23:43:31.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.071+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94847199048064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1a800 con 0x564355a1a000 2024-09-13T23:43:31.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.071+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94847199048064] v33'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.071+0000 7ff4c4e99f40 20 remove_watcher() i=5 2024-09-13T23:43:31.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.071+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94847199049216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1bc00 con 0x564355a1a000 2024-09-13T23:43:31.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94847199049216] v33'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b37440 con 0x564355a1a000 2024-09-13T23:43:31.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 20 remove_watcher() i=6 2024-09-13T23:43:31.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94847199050368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x564355a1b800 con 0x564355a1a400 2024-09-13T23:43:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c3e53700 1 -- 172.21.15.167:0/4272765924 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94847199050368] v33'101 uv99 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564355b36b40 con 0x564355a1a400 2024-09-13T23:43:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 20 remove_watcher() i=7 2024-09-13T23:43:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564355a81400 msgr2=0x5643559c0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564355a81400 0x5643559c0100 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564355a81000 msgr2=0x5643559bfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564355a81000 0x5643559bfb80 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564355a1a400 msgr2=0x5643559c0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564355a1a400 0x5643559c0c00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564355a1a000 msgr2=0x5643559c0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564355a1a000 0x5643559c0680 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355a80c00 msgr2=0x5643559bf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355a80c00 0x5643559bf600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x5643559b9920 tx=0x5643557dd800 comp rx=0 tx=0).stop 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 msgr2=0x5643559beb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 0x5643559beb00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x564355a04360 tx=0x5643557ddbc0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c2e51700 1 -- 172.21.15.167:0/4272765924 reap_dead start 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 shutdown_connections 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/4272765924 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9800 0x5643559beb00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 >> 172.21.15.167:0/4272765924 conn(0x5643556d9400 msgr2=0x5643557d9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 shutdown_connections 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/4272765924 wait complete. 2024-09-13T23:43:31.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/928901722 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355bd8000 msgr2=0x564355bde000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/928901722 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355bd8000 0x564355bde000 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x564355ad9d70 tx=0x564355a397a0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/928901722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 msgr2=0x5643559c1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/928901722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 0x5643559c1180 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x564355b587e0 tx=0x564355b5a300 comp rx=0 tx=0).stop 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/928901722 shutdown_connections 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/928901722 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355bd8000 0x564355bde000 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/928901722 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564355a1cc00 0x5643559c1180 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/928901722 >> 172.21.15.167:0/928901722 conn(0x564355a1dc00 msgr2=0x5643557da000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/928901722 shutdown_connections 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/928901722 wait complete. 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1556175037 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355a80000 msgr2=0x5643559bf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/1556175037 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355a80000 0x5643559bf080 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x564355a054d0 tx=0x564355a38240 comp rx=0 tx=0).stop 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1556175037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 msgr2=0x5643559be580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/1556175037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 0x5643559be580 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x5643564a3e60 tx=0x5643557dd920 comp rx=0 tx=0).stop 2024-09-13T23:43:31.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1556175037 shutdown_connections 2024-09-13T23:43:31.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/1556175037 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564355a80000 0x5643559bf080 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 --2- 172.21.15.167:0/1556175037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5643556d9000 0x5643559be580 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1556175037 >> 172.21.15.167:0/1556175037 conn(0x5643556d8c00 msgr2=0x5643557d9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1556175037 shutdown_connections 2024-09-13T23:43:31.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.075+0000 7ff4c4e99f40 1 -- 172.21.15.167:0/1556175037 wait complete. 2024-09-13T23:43:31.090 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:31.090 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo", 2024-09-13T23:43:31.090 INFO:teuthology.orchestra.run.smithi167.stdout: "num_shards": 11, 2024-09-13T23:43:31.090 INFO:teuthology.orchestra.run.smithi167.stdout: "tenant": "", 2024-09-13T23:43:31.090 INFO:teuthology.orchestra.run.smithi167.stdout: "zonegroup": "363ea8c8-5049-424d-855a-cd6979bf7acf", 2024-09-13T23:43:31.090 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_rule": "default-placement", 2024-09-13T23:43:31.090 INFO:teuthology.orchestra.run.smithi167.stdout: "explicit_placement": { 2024-09-13T23:43:31.090 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": "", 2024-09-13T23:43:31.090 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "", 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": "" 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1", 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "marker": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1", 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "index_type": "Normal", 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "versioned": false, 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "versioning_enabled": false, 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "object_lock_enabled": false, 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_enabled": false, 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1", 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "mtime": "2024-09-13T23:43:29.857870Z", 2024-09-13T23:43:31.091 INFO:teuthology.orchestra.run.smithi167.stdout: "creation_time": "2024-09-13T23:43:29.849424Z", 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: "usage": { 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: "rgw.main": { 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: "size": 3, 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: "size_actual": 4096, 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: "size_utilized": 3, 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: "size_kb": 1, 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: "size_kb_actual": 4, 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: "size_kb_utilized": 1, 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: "num_objects": 1 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:31.092 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:31.093 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:31.093 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:31.093 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:31.093 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:31.093 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:31.093 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:31.093 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '363ea8c8-5049-424d-855a-cd6979bf7acf', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1', 'marker': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-13T23:43:29.857870Z', 'creation_time': '2024-09-13T23:43:29.849424Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-09-13T23:43:31.093 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-09-13T23:43:31.093 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-09-13T23:43:31.094 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user stats --uid foo --sync-stats 2024-09-13T23:43:31.133 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:31.133 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:31.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.123+0000 7f25058fcf40 1 Processor -- start 2024-09-13T23:43:31.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.123+0000 7f25058fcf40 1 -- start start 2024-09-13T23:43:31.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.123+0000 7f25058fcf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x556d4280b000 0x556d42af0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.123+0000 7f25058fcf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x556d428cf500 con 0x556d4280b400 2024-09-13T23:43:31.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.123+0000 7f25058fcf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x556d42898a00 con 0x556d4280b000 2024-09-13T23:43:31.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25040b5700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x556d4280b000 0x556d42af0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25038b4700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x556d4280b400 0x556d4290a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45450/0 (socket says 172.21.15.167:45450) 2024-09-13T23:43:31.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25038b4700 1 -- 172.21.15.167:0/3201962738 learned_addr learned my addr 172.21.15.167:0/3201962738 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:31.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25030b3700 1 -- 172.21.15.167:0/3201962738 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333522481 0 0) 0x556d42966000 con 0x556d4280b400 2024-09-13T23:43:31.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25030b3700 1 -- 172.21.15.167:0/3201962738 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x556d428cf680 con 0x556d4280b400 2024-09-13T23:43:31.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25040b5700 1 -- 172.21.15.167:0/3201962738 >> v1:172.21.15.12:6789/0 conn(0x556d4280b400 legacy=0x556d4290a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25040b5700 1 -- 172.21.15.167:0/3201962738 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d4281dd40 con 0x556d4280b000 2024-09-13T23:43:31.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25040b5700 1 --2- 172.21.15.167:0/3201962738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x556d42aeba70 tx=0x556d4290f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c23fe1e8d7149e72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25030b3700 1 -- 172.21.15.167:0/3201962738 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d428af4a0 con 0x556d4280b000 2024-09-13T23:43:31.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25030b3700 1 -- 172.21.15.167:0/3201962738 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d428cf500 con 0x556d4280b000 2024-09-13T23:43:31.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25030b3700 1 -- 172.21.15.167:0/3201962738 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d428af600 con 0x556d4280b000 2024-09-13T23:43:31.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25058fcf40 1 -- 172.21.15.167:0/3201962738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 msgr2=0x556d42af0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25058fcf40 1 --2- 172.21.15.167:0/3201962738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x556d42aeba70 tx=0x556d4290f980 comp rx=0 tx=0).stop 2024-09-13T23:43:31.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25058fcf40 1 -- 172.21.15.167:0/3201962738 shutdown_connections 2024-09-13T23:43:31.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25058fcf40 1 --2- 172.21.15.167:0/3201962738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25058fcf40 1 -- 172.21.15.167:0/3201962738 >> 172.21.15.167:0/3201962738 conn(0x556d4280ac00 msgr2=0x556d4290a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25058fcf40 1 -- 172.21.15.167:0/3201962738 shutdown_connections 2024-09-13T23:43:31.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25058fcf40 1 -- 172.21.15.167:0/3201962738 wait complete. 2024-09-13T23:43:31.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25058fcf40 1 Processor -- start 2024-09-13T23:43:31.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25058fcf40 1 -- start start 2024-09-13T23:43:31.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25058fcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25058fcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d42898500 con 0x556d4280b000 2024-09-13T23:43:31.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.127+0000 7f25040b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25040b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52028/0 (socket says 172.21.15.167:52028) 2024-09-13T23:43:31.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25040b5700 1 -- 172.21.15.167:0/1652164039 learned_addr learned my addr 172.21.15.167:0/1652164039 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25040b5700 1 -- 172.21.15.167:0/1652164039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d4281dba0 con 0x556d4280b000 2024-09-13T23:43:31.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25040b5700 1 --2- 172.21.15.167:0/1652164039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x556d435d5e00 tx=0x556d4290f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186620fa50661297 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25028b2700 1 -- 172.21.15.167:0/1652164039 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d428af760 con 0x556d4280b000 2024-09-13T23:43:31.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25028b2700 1 -- 172.21.15.167:0/1652164039 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d428cf980 con 0x556d4280b000 2024-09-13T23:43:31.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25028b2700 1 -- 172.21.15.167:0/1652164039 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d428af4a0 con 0x556d4280b000 2024-09-13T23:43:31.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25058fcf40 1 -- 172.21.15.167:0/1652164039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 msgr2=0x556d42af0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1652164039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x556d435d5e00 tx=0x556d4290f800 comp rx=0 tx=0).stop 2024-09-13T23:43:31.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25058fcf40 1 -- 172.21.15.167:0/1652164039 shutdown_connections 2024-09-13T23:43:31.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1652164039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25058fcf40 1 -- 172.21.15.167:0/1652164039 >> 172.21.15.167:0/1652164039 conn(0x556d4280ac00 msgr2=0x556d4290b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25058fcf40 1 -- 172.21.15.167:0/1652164039 shutdown_connections 2024-09-13T23:43:31.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25058fcf40 1 -- 172.21.15.167:0/1652164039 wait complete. 2024-09-13T23:43:31.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25058fcf40 1 Processor -- start 2024-09-13T23:43:31.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25058fcf40 1 -- start start 2024-09-13T23:43:31.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25058fcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25058fcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d428983c0 con 0x556d4280b000 2024-09-13T23:43:31.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25040b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25040b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52032/0 (socket says 172.21.15.167:52032) 2024-09-13T23:43:31.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25040b5700 1 -- 172.21.15.167:0/3117264123 learned_addr learned my addr 172.21.15.167:0/3117264123 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.131+0000 7f25040b5700 1 -- 172.21.15.167:0/3117264123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d4281da00 con 0x556d4280b000 2024-09-13T23:43:31.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25040b5700 1 --2- 172.21.15.167:0/3117264123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x556d435d5ef0 tx=0x556d4290f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25010af700 1 -- 172.21.15.167:0/3117264123 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d428af600 con 0x556d4280b000 2024-09-13T23:43:31.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25010af700 1 -- 172.21.15.167:0/3117264123 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d428cfb00 con 0x556d4280b000 2024-09-13T23:43:31.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25058fcf40 1 -- 172.21.15.167:0/3117264123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556d4281d860 con 0x556d4280b000 2024-09-13T23:43:31.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25010af700 1 -- 172.21.15.167:0/3117264123 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d428af8c0 con 0x556d4280b000 2024-09-13T23:43:31.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25058fcf40 1 -- 172.21.15.167:0/3117264123 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d4281d6c0 con 0x556d4280b000 2024-09-13T23:43:31.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25058fcf40 1 Processor -- start 2024-09-13T23:43:31.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25058fcf40 1 -- start start 2024-09-13T23:43:31.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25058fcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 0x556d42af0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25058fcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d42898000 con 0x556d4280b800 2024-09-13T23:43:31.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25048b6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 0x556d42af0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25048b6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 0x556d42af0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52034/0 (socket says 172.21.15.167:52034) 2024-09-13T23:43:31.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25048b6700 1 -- 172.21.15.167:0/1010132221 learned_addr learned my addr 172.21.15.167:0/1010132221 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25010af700 1 -- 172.21.15.167:0/3117264123 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x556d42904b00 con 0x556d4280b000 2024-09-13T23:43:31.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25048b6700 1 -- 172.21.15.167:0/1010132221 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d4281c000 con 0x556d4280b800 2024-09-13T23:43:31.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25048b6700 1 --2- 172.21.15.167:0/1010132221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 0x556d42af0b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x556d42b25380 tx=0x556d42b82540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cde777b8fa75b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25010af700 1 --2- 172.21.15.167:0/3117264123 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42bc4000 0x556d42af1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f24fe8aa700 1 -- 172.21.15.167:0/1010132221 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d428af600 con 0x556d4280b800 2024-09-13T23:43:31.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25010af700 1 -- 172.21.15.167:0/3117264123 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x556d42871180 con 0x556d4280b000 2024-09-13T23:43:31.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f24fe8aa700 1 -- 172.21.15.167:0/1010132221 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d428cfb00 con 0x556d4280b800 2024-09-13T23:43:31.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f24fe8aa700 1 -- 172.21.15.167:0/1010132221 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d428af8c0 con 0x556d4280b800 2024-09-13T23:43:31.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.135+0000 7f25040b5700 1 --2- 172.21.15.167:0/3117264123 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42bc4000 0x556d42af1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25058fcf40 1 -- 172.21.15.167:0/1010132221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 msgr2=0x556d42af0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1010132221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 0x556d42af0b00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x556d42b25380 tx=0x556d42b82540 comp rx=0 tx=0).stop 2024-09-13T23:43:31.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25040b5700 1 --2- 172.21.15.167:0/3117264123 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42bc4000 0x556d42af1080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x556d42b25680 tx=0x556d42b82120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25058fcf40 1 -- 172.21.15.167:0/1010132221 shutdown_connections 2024-09-13T23:43:31.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1010132221 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 0x556d42af0b00 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25058fcf40 1 -- 172.21.15.167:0/1010132221 >> 172.21.15.167:0/1010132221 conn(0x556d4280b400 msgr2=0x556d4290b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25058fcf40 1 -- 172.21.15.167:0/1010132221 shutdown_connections 2024-09-13T23:43:31.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25058fcf40 1 -- 172.21.15.167:0/1010132221 wait complete. 2024-09-13T23:43:31.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25058fcf40 1 Processor -- start 2024-09-13T23:43:31.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25058fcf40 1 -- start start 2024-09-13T23:43:31.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25058fcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 0x556d42af0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25058fcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d42830640 con 0x556d4280b800 2024-09-13T23:43:31.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25048b6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 0x556d42af0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25048b6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 0x556d42af0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52048/0 (socket says 172.21.15.167:52048) 2024-09-13T23:43:31.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 learned_addr learned my addr 172.21.15.167:0/1615415131 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d4281cea0 con 0x556d4280b800 2024-09-13T23:43:31.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.139+0000 7f25048b6700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 0x556d42af0b00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x556d42b24e40 tx=0x556d4290fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f24fd0a7700 1 -- 172.21.15.167:0/1615415131 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d428af760 con 0x556d4280b800 2024-09-13T23:43:31.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f24fd0a7700 1 -- 172.21.15.167:0/1615415131 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d42bdc000 con 0x556d4280b800 2024-09-13T23:43:31.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556d4281c820 con 0x556d4280b800 2024-09-13T23:43:31.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f24fd0a7700 1 -- 172.21.15.167:0/1615415131 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d428af600 con 0x556d4280b800 2024-09-13T23:43:31.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d4281c9c0 con 0x556d4280b800 2024-09-13T23:43:31.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f24fd0a7700 1 -- 172.21.15.167:0/1615415131 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x556d42904dc0 con 0x556d4280b800 2024-09-13T23:43:31.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f24fd0a7700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42bc4c00 0x556d42af1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f24fd0a7700 1 -- 172.21.15.167:0/1615415131 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x556d42871340 con 0x556d4280b800 2024-09-13T23:43:31.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f25038b4700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42bc4c00 0x556d42af1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x556d42bc5000 0x556d42af1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5400 con 0x556d42bc5000 2024-09-13T23:43:31.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f25048b6700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x556d42bc5000 0x556d42af1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f25038b4700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42bc4c00 0x556d42af1600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x556d435d4030 tx=0x556d42b83f80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.143+0000 7f25048b6700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x556d42bc5000 0x556d42af1b80 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 20 realm 2024-09-13T23:43:31.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5000 2024-09-13T23:43:31.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5c00 con 0x556d42bc5000 2024-09-13T23:43:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x556d42bc5400 0x556d42af2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5400 2024-09-13T23:43:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25040b5700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x556d42bc5400 0x556d42af2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25040b5700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x556d42bc5400 0x556d42af2100 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:31.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5c00 con 0x556d42bc5400 2024-09-13T23:43:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.147+0000 7f25058fcf40 20 searching for the correct realm 2024-09-13T23:43:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x556d42b2e000 0x556d42af2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2e400 con 0x556d42b2e000 2024-09-13T23:43:31.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25038b4700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x556d42b2e000 0x556d42af2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25038b4700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x556d42b2e000 0x556d42af2680 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2e400 con 0x556d42bc5400 2024-09-13T23:43:31.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25040b5700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x556d42b2e400 0x556d42af2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e400 2024-09-13T23:43:31.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25048b6700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x556d42b2e400 0x556d42af2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25048b6700 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x556d42b2e400 0x556d42af2c00 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5000 2024-09-13T23:43:31.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.151+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5000 2024-09-13T23:43:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e000 2024-09-13T23:43:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5000 2024-09-13T23:43:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5400 2024-09-13T23:43:31.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5400 2024-09-13T23:43:31.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5000 2024-09-13T23:43:31.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e400 2024-09-13T23:43:31.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.155+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5000 2024-09-13T23:43:31.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.159+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.159+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e400 2024-09-13T23:43:31.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.159+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.159+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5400 2024-09-13T23:43:31.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.159+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.159+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e000 2024-09-13T23:43:31.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.159+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.159+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5000 2024-09-13T23:43:31.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.159+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.159+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5400 2024-09-13T23:43:31.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.159+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.159+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e000 2024-09-13T23:43:31.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5000 2024-09-13T23:43:31.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5000 2024-09-13T23:43:31.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5000 2024-09-13T23:43:31.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e000 2024-09-13T23:43:31.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e400 2024-09-13T23:43:31.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5000 2024-09-13T23:43:31.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.163+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e400 2024-09-13T23:43:31.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.167+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.167+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e000 2024-09-13T23:43:31.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.167+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.167+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e400 2024-09-13T23:43:31.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.167+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.167+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5400 2024-09-13T23:43:31.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.167+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.167+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5400 2024-09-13T23:43:31.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.167+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.167+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e000 2024-09-13T23:43:31.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5400 2024-09-13T23:43:31.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42bc5400 2024-09-13T23:43:31.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25058fcf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:31.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25058fcf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:31.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25058fcf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:31.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25058fcf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:31.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2e800 con 0x556d42bc5000 2024-09-13T23:43:31.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25058fcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2ec00 con 0x556d42bc5400 2024-09-13T23:43:31.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25058fcf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:31.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.171+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2f000 con 0x556d42bc5400 2024-09-13T23:43:31.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:31.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 20 zone default found 2024-09-13T23:43:31.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 4 Realm: () 2024-09-13T23:43:31.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:31.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:31.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:31.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 20 zonegroup default 2024-09-13T23:43:31.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2f400 con 0x556d42b2e400 2024-09-13T23:43:31.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2f800 con 0x556d42b2e000 2024-09-13T23:43:31.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2fc00 con 0x556d42b2e000 2024-09-13T23:43:31.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 20 started sync module instance, tier type = 2024-09-13T23:43:31.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:31.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.175+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2f800 con 0x556d42bc5400 2024-09-13T23:43:31.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.179+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.179+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93927758527232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2fc00 con 0x556d42bc5400 2024-09-13T23:43:31.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.179+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2ec00 con 0x556d42bc5000 2024-09-13T23:43:31.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.183+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.183+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93927758528384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2f000 con 0x556d42bc5000 2024-09-13T23:43:31.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.183+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 93927758527232] v33'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.183+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2f800 con 0x556d42bc5400 2024-09-13T23:43:31.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.183+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.183+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93927758529536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5c00 con 0x556d42bc5400 2024-09-13T23:43:31.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.183+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e400 2024-09-13T23:43:31.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.183+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93927758528384] v33'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.187+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 93927758529536] v33'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.187+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.187+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93927758530688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2e800 con 0x556d42b2e400 2024-09-13T23:43:31.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.187+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2f400 con 0x556d42bc5400 2024-09-13T23:43:31.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.191+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 93927758530688] v33'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.191+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.191+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93927758531840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2ec00 con 0x556d42bc5400 2024-09-13T23:43:31.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.191+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2f000 con 0x556d42b2e000 2024-09-13T23:43:31.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.191+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.191+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93927758532992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b30000 con 0x556d42b2e000 2024-09-13T23:43:31.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.191+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93927758531840] v33'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.191+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b30400 con 0x556d42b2e000 2024-09-13T23:43:31.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93927758532992] v33'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93927758534144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b30800 con 0x556d42b2e000 2024-09-13T23:43:31.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b30c00 con 0x556d42b2e400 2024-09-13T23:43:31.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'102 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93927758535296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b31000 con 0x556d42b2e400 2024-09-13T23:43:31.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25058fcf40 20 add_watcher() i=0 2024-09-13T23:43:31.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25058fcf40 20 add_watcher() i=1 2024-09-13T23:43:31.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25058fcf40 20 add_watcher() i=2 2024-09-13T23:43:31.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25058fcf40 20 add_watcher() i=3 2024-09-13T23:43:31.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25058fcf40 20 add_watcher() i=4 2024-09-13T23:43:31.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25058fcf40 20 add_watcher() i=5 2024-09-13T23:43:31.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93927758534144] v33'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.195+0000 7f25058fcf40 20 add_watcher() i=6 2024-09-13T23:43:31.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.199+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 93927758535296] v33'103 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.199+0000 7f25058fcf40 20 add_watcher() i=7 2024-09-13T23:43:31.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.199+0000 7f25058fcf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:31.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.199+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b31400 con 0x556d42bc5000 2024-09-13T23:43:31.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.199+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.199+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93927760232448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b31800 con 0x556d42bc5000 2024-09-13T23:43:31.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93927760232448] v33'102 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f24eb083700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:31.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:31.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:31.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 1 Processor -- start 2024-09-13T23:43:31.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 1 -- start start 2024-09-13T23:43:31.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 0x556d42af3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d42830780 con 0x556d42b30c00 2024-09-13T23:43:31.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25038b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 0x556d42af3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25038b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 0x556d42af3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52058/0 (socket says 172.21.15.167:52058) 2024-09-13T23:43:31.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25038b4700 1 -- 172.21.15.167:0/2156835676 learned_addr learned my addr 172.21.15.167:0/2156835676 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25038b4700 1 -- 172.21.15.167:0/2156835676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d4281cb60 con 0x556d42b30c00 2024-09-13T23:43:31.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25038b4700 1 --2- 172.21.15.167:0/2156835676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 0x556d42af3180 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x556d42c920c0 tx=0x556d42c94000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00635ea6f660b91 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f24ea882700 1 -- 172.21.15.167:0/2156835676 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d42c98000 con 0x556d42b30c00 2024-09-13T23:43:31.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f24ea882700 1 -- 172.21.15.167:0/2156835676 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d42bdc480 con 0x556d42b30c00 2024-09-13T23:43:31.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f24ea882700 1 -- 172.21.15.167:0/2156835676 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d42c98160 con 0x556d42b30c00 2024-09-13T23:43:31.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 1 -- 172.21.15.167:0/2156835676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 msgr2=0x556d42af3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 1 --2- 172.21.15.167:0/2156835676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 0x556d42af3180 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x556d42c920c0 tx=0x556d42c94000 comp rx=0 tx=0).stop 2024-09-13T23:43:31.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 1 -- 172.21.15.167:0/2156835676 shutdown_connections 2024-09-13T23:43:31.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 1 --2- 172.21.15.167:0/2156835676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 0x556d42af3180 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 1 -- 172.21.15.167:0/2156835676 >> 172.21.15.167:0/2156835676 conn(0x556d42b31c00 msgr2=0x556d4290c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 1 -- 172.21.15.167:0/2156835676 shutdown_connections 2024-09-13T23:43:31.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.203+0000 7f25058fcf40 1 -- 172.21.15.167:0/2156835676 wait complete. 2024-09-13T23:43:31.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f25058fcf40 1 Processor -- start 2024-09-13T23:43:31.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f25058fcf40 1 -- start start 2024-09-13T23:43:31.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f25058fcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 0x556d42af3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f25058fcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d42898000 con 0x556d42b30c00 2024-09-13T23:43:31.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f25038b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 0x556d42af3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f25038b4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 0x556d42af3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52060/0 (socket says 172.21.15.167:52060) 2024-09-13T23:43:31.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f25038b4700 1 -- 172.21.15.167:0/2903591943 learned_addr learned my addr 172.21.15.167:0/2903591943 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f25038b4700 1 -- 172.21.15.167:0/2903591943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d4281cd00 con 0x556d42b30c00 2024-09-13T23:43:31.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f25038b4700 1 --2- 172.21.15.167:0/2903591943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 0x556d42af3180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x556d42c93380 tx=0x556d42c942a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f24e907f700 1 -- 172.21.15.167:0/2903591943 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d42c982c0 con 0x556d42b30c00 2024-09-13T23:43:31.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f24e907f700 1 -- 172.21.15.167:0/2903591943 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556d42bdc600 con 0x556d42b30c00 2024-09-13T23:43:31.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f25058fcf40 1 -- 172.21.15.167:0/2903591943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556d4281c340 con 0x556d42b30c00 2024-09-13T23:43:31.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f24e907f700 1 -- 172.21.15.167:0/2903591943 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x556d42c98000 con 0x556d42b30c00 2024-09-13T23:43:31.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.207+0000 7f25058fcf40 1 -- 172.21.15.167:0/2903591943 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d4281cb60 con 0x556d42b30c00 2024-09-13T23:43:31.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f24e907f700 1 -- 172.21.15.167:0/2903591943 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x556d42905b80 con 0x556d42b30c00 2024-09-13T23:43:31.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f24e907f700 1 --2- 172.21.15.167:0/2903591943 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42d10000 0x556d42d16000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f24e907f700 1 -- 172.21.15.167:0/2903591943 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x556d42871500 con 0x556d42b30c00 2024-09-13T23:43:31.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f25058fcf40 5 note: GC not initialized 2024-09-13T23:43:31.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f25048b6700 1 --2- 172.21.15.167:0/2903591943 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42d10000 0x556d42d16000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f25048b6700 1 --2- 172.21.15.167:0/2903591943 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42d10000 0x556d42d16000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x556d42c137d0 tx=0x556d42b833e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f24e687a700 20 reqs_thread_entry: start 2024-09-13T23:43:31.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f25058fcf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:31.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f25058fcf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:31.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f24e4876700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:01 2024 2024-09-13T23:43:31.222 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:31.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f24e4054700 20 reqs_thread_entry: start 2024-09-13T23:43:31.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b30000 con 0x556d42bc5000 2024-09-13T23:43:31.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.211+0000 7f25058fcf40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:31.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b30400 con 0x556d42b2e400 2024-09-13T23:43:31.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2e800 con 0x556d42b2e000 2024-09-13T23:43:31.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(64 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:31.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-13T23:43:31.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 4.a 4:517e2946:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2f000 con 0x556d42bc5400 2024-09-13T23:43:31.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:66 4.8 4:10b47b19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e000 2024-09-13T23:43:31.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 4.2e 4:75bb53d2:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42d10400 con 0x556d42b2e000 2024-09-13T23:43:31.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 4.26 4:6424e2c9:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42d10800 con 0x556d42bc5000 2024-09-13T23:43:31.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 4.c 4:32a29db1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42d10c00 con 0x556d42bc5000 2024-09-13T23:43:31.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 4.28 4:162fa60a:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42d11000 con 0x556d42bc5400 2024-09-13T23:43:31.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:71 4.2c 4:355fcb42:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42d11400 con 0x556d42bc5000 2024-09-13T23:43:31.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 4.2f 4:f64e09ba:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42d11800 con 0x556d42bc5000 2024-09-13T23:43:31.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:73 4.3a 4:5ee337b5:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42d11c00 con 0x556d42b2e000 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:74 4.2c 4:3559f32d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2fc00 con 0x556d42bc5000 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:75 4.1 4:8131d739:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2f800 con 0x556d42b2e400 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(70 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.5 [call out=124b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(66 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(68 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(67 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(69 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(71 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(72 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(75 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(73 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(74 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.9 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.215+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2f400 con 0x556d42b2e400 2024-09-13T23:43:31.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.219+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(76 foo.buckets [call] v33'15 uv15 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.219+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:77 8.9 8:96d98480:users.uid::foo.buckets:head [call user.complete_stats_sync in=37b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2ec00 con 0x556d42b2e400 2024-09-13T23:43:31.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.219+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(77 foo.buckets [call] v33'16 uv16 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.219+0000 7f25058fcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.219+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:78 8.18 8:1e6ca5ed:users.uid::foo:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2f000 con 0x556d42bc5000 2024-09-13T23:43:31.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.219+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(78 foo [stat out=16b,read 0~357 out=357b] v0'0 uv11 ondisk = 0) v8 ==== 189+0+373 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.219+0000 7f25058fcf40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:31.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.223+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:79 8.9 8:96d98480:users.uid::foo.buckets:head [call user.get_header in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5c00 con 0x556d42b2e400 2024-09-13T23:43:31.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.223+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(79 foo.buckets [call out=58b] v0'0 uv16 ondisk = 0) v8 ==== 155+0+58 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.223+0000 7f24eb083700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:31.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.223+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:80 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93927760232448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42d11800 con 0x556d42bc5000 2024-09-13T23:43:31.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.227+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(80 data_loggenerations_metadata [watch unwatch cookie 93927760232448] v33'103 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.227+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:81 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93927758527232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42d10400 con 0x556d42bc5400 2024-09-13T23:43:31.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.227+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(81 notify.0 [watch unwatch cookie 93927758527232] v33'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.227+0000 7f25058fcf40 20 remove_watcher() i=0 2024-09-13T23:43:31.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.227+0000 7f25058fcf40 2 removed watcher, disabling cache 2024-09-13T23:43:31.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.227+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:82 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93927758528384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42d11c00 con 0x556d42bc5000 2024-09-13T23:43:31.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.231+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(82 notify.1 [watch unwatch cookie 93927758528384] v33'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42bc5000 2024-09-13T23:43:31.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.231+0000 7f25058fcf40 20 remove_watcher() i=1 2024-09-13T23:43:31.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.231+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:83 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93927758529536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b31800 con 0x556d42bc5400 2024-09-13T23:43:31.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.231+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(83 notify.2 [watch unwatch cookie 93927758529536] v33'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.231+0000 7f25058fcf40 20 remove_watcher() i=2 2024-09-13T23:43:31.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.231+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:84 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93927758530688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b30000 con 0x556d42b2e400 2024-09-13T23:43:31.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.235+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(84 notify.3 [watch unwatch cookie 93927758530688] v33'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.235+0000 7f25058fcf40 20 remove_watcher() i=3 2024-09-13T23:43:31.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.235+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:85 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93927758531840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b30400 con 0x556d42bc5400 2024-09-13T23:43:31.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.239+0000 7f25040b5700 1 -- 172.21.15.167:0/1615415131 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(85 notify.4 [watch unwatch cookie 93927758531840] v33'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d200 con 0x556d42bc5400 2024-09-13T23:43:31.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.239+0000 7f25058fcf40 20 remove_watcher() i=4 2024-09-13T23:43:31.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.239+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:86 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93927758532992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5c00 con 0x556d42b2e000 2024-09-13T23:43:31.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.239+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(86 notify.5 [watch unwatch cookie 93927758532992] v33'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.239+0000 7f25058fcf40 20 remove_watcher() i=5 2024-09-13T23:43:31.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.239+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:87 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93927758534144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42bc5800 con 0x556d42b2e000 2024-09-13T23:43:31.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(87 notify.6 [watch unwatch cookie 93927758534144] v33'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6d440 con 0x556d42b2e000 2024-09-13T23:43:31.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25058fcf40 20 remove_watcher() i=6 2024-09-13T23:43:31.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:88 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93927758535296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x556d42b2fc00 con 0x556d42b2e400 2024-09-13T23:43:31.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25048b6700 1 -- 172.21.15.167:0/1615415131 <== osd.2 v2:172.21.15.167:6808/1180369139 18 ==== osd_op_reply(88 notify.7 [watch unwatch cookie 93927758535296] v33'104 uv102 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x556d42c6cb40 con 0x556d42b2e400 2024-09-13T23:43:31.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25058fcf40 20 remove_watcher() i=7 2024-09-13T23:43:31.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x556d42bc5400 msgr2=0x556d42af2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x556d42bc5400 0x556d42af2100 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x556d42bc5000 msgr2=0x556d42af1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x556d42bc5000 0x556d42af1b80 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x556d42b2e400 msgr2=0x556d42af2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x556d42b2e400 0x556d42af2c00 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x556d42b2e000 msgr2=0x556d42af2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.243+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x556d42b2e000 0x556d42af2680 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42bc4c00 msgr2=0x556d42af1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42bc4c00 0x556d42af1600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x556d435d4030 tx=0x556d42b83f80 comp rx=0 tx=0).stop 2024-09-13T23:43:31.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 msgr2=0x556d42af0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 0x556d42af0b00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x556d42b24e40 tx=0x556d4290fb60 comp rx=0 tx=0).stop 2024-09-13T23:43:31.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25038b4700 1 -- 172.21.15.167:0/1615415131 reap_dead start 2024-09-13T23:43:31.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 shutdown_connections 2024-09-13T23:43:31.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 --2- 172.21.15.167:0/1615415131 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b800 0x556d42af0b00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 >> 172.21.15.167:0/1615415131 conn(0x556d4280b400 msgr2=0x556d4290b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 shutdown_connections 2024-09-13T23:43:31.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/1615415131 wait complete. 2024-09-13T23:43:31.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/2903591943 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42d10000 msgr2=0x556d42d16000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 --2- 172.21.15.167:0/2903591943 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42d10000 0x556d42d16000 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x556d42c137d0 tx=0x556d42b833e0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/2903591943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 msgr2=0x556d42af3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 --2- 172.21.15.167:0/2903591943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 0x556d42af3180 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x556d42c93380 tx=0x556d42c942a0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/2903591943 shutdown_connections 2024-09-13T23:43:31.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 --2- 172.21.15.167:0/2903591943 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42d10000 0x556d42d16000 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 --2- 172.21.15.167:0/2903591943 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d42b30c00 0x556d42af3180 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/2903591943 >> 172.21.15.167:0/2903591943 conn(0x556d42b31c00 msgr2=0x556d4290c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/2903591943 shutdown_connections 2024-09-13T23:43:31.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/2903591943 wait complete. 2024-09-13T23:43:31.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/3117264123 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42bc4000 msgr2=0x556d42af1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 --2- 172.21.15.167:0/3117264123 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42bc4000 0x556d42af1080 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x556d42b25680 tx=0x556d42b82120 comp rx=0 tx=0).stop 2024-09-13T23:43:31.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/3117264123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 msgr2=0x556d42af0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 --2- 172.21.15.167:0/3117264123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x556d435d5ef0 tx=0x556d4290f980 comp rx=0 tx=0).stop 2024-09-13T23:43:31.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/3117264123 shutdown_connections 2024-09-13T23:43:31.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 --2- 172.21.15.167:0/3117264123 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x556d42bc4000 0x556d42af1080 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 --2- 172.21.15.167:0/3117264123 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x556d4280b000 0x556d42af0580 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/3117264123 >> 172.21.15.167:0/3117264123 conn(0x556d4280ac00 msgr2=0x556d4290b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/3117264123 shutdown_connections 2024-09-13T23:43:31.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.247+0000 7f25058fcf40 1 -- 172.21.15.167:0/3117264123 wait complete. 2024-09-13T23:43:31.259 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:31.259 INFO:teuthology.orchestra.run.smithi167.stdout: "stats": { 2024-09-13T23:43:31.259 INFO:teuthology.orchestra.run.smithi167.stdout: "size": 3, 2024-09-13T23:43:31.260 INFO:teuthology.orchestra.run.smithi167.stdout: "size_actual": 4096, 2024-09-13T23:43:31.260 INFO:teuthology.orchestra.run.smithi167.stdout: "size_kb": 1, 2024-09-13T23:43:31.260 INFO:teuthology.orchestra.run.smithi167.stdout: "size_kb_actual": 4, 2024-09-13T23:43:31.260 INFO:teuthology.orchestra.run.smithi167.stdout: "num_objects": 1 2024-09-13T23:43:31.260 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:31.260 INFO:teuthology.orchestra.run.smithi167.stdout: "last_stats_sync": "2024-09-13T23:43:31.223873Z", 2024-09-13T23:43:31.260 INFO:teuthology.orchestra.run.smithi167.stdout: "last_stats_update": "2024-09-13T23:43:31.221639Z" 2024-09-13T23:43:31.260 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:31.262 DEBUG:tasks.util.rgw: json result: {'stats': {'size': 3, 'size_actual': 4096, 'size_kb': 1, 'size_kb_actual': 4, 'num_objects': 1}, 'last_stats_sync': '2024-09-13T23:43:31.223873Z', 'last_stats_update': '2024-09-13T23:43:31.221639Z'} 2024-09-13T23:43:31.271 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:31.271 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-09-13T23:43:31.271 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-09-13T23:43:31.271 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid foo --bucket myfoo 2024-09-13T23:43:31.311 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:31.311 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:31.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa334c20f40 1 Processor -- start 2024-09-13T23:43:31.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa334c20f40 1 -- start start 2024-09-13T23:43:31.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa334c20f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f300f05000 0x55f3011ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa334c20f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f300fc9500 con 0x55f300f05400 2024-09-13T23:43:31.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa334c20f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f300f92a00 con 0x55f300f05000 2024-09-13T23:43:31.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa3333d9700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f300f05000 0x55f3011ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa332bd8700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55f300f05400 0x55f301004800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45460/0 (socket says 172.21.15.167:45460) 2024-09-13T23:43:31.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa332bd8700 1 -- 172.21.15.167:0/3458011739 learned_addr learned my addr 172.21.15.167:0/3458011739 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:31.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa3323d7700 1 -- 172.21.15.167:0/3458011739 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781621012 0 0) 0x55f301060000 con 0x55f300f05400 2024-09-13T23:43:31.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa3323d7700 1 -- 172.21.15.167:0/3458011739 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55f300fc9680 con 0x55f300f05400 2024-09-13T23:43:31.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa3333d9700 1 -- 172.21.15.167:0/3458011739 >> v1:172.21.15.12:6789/0 conn(0x55f300f05400 legacy=0x55f301004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa3333d9700 1 -- 172.21.15.167:0/3458011739 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f300f17d40 con 0x55f300f05000 2024-09-13T23:43:31.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.303+0000 7fa3333d9700 1 --2- 172.21.15.167:0/3458011739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55f3011e5a70 tx=0x55f301009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bed97b74772035 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa3323d7700 1 -- 172.21.15.167:0/3458011739 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f300fa94a0 con 0x55f300f05000 2024-09-13T23:43:31.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa3323d7700 1 -- 172.21.15.167:0/3458011739 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f300fc9680 con 0x55f300f05000 2024-09-13T23:43:31.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa3323d7700 1 -- 172.21.15.167:0/3458011739 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f300fa9600 con 0x55f300f05000 2024-09-13T23:43:31.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa334c20f40 1 -- 172.21.15.167:0/3458011739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 msgr2=0x55f3011ea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa334c20f40 1 --2- 172.21.15.167:0/3458011739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x55f3011e5a70 tx=0x55f301009980 comp rx=0 tx=0).stop 2024-09-13T23:43:31.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa334c20f40 1 -- 172.21.15.167:0/3458011739 shutdown_connections 2024-09-13T23:43:31.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa334c20f40 1 --2- 172.21.15.167:0/3458011739 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa334c20f40 1 -- 172.21.15.167:0/3458011739 >> 172.21.15.167:0/3458011739 conn(0x55f300f04c00 msgr2=0x55f301004000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa334c20f40 1 -- 172.21.15.167:0/3458011739 shutdown_connections 2024-09-13T23:43:31.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa334c20f40 1 -- 172.21.15.167:0/3458011739 wait complete. 2024-09-13T23:43:31.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa334c20f40 1 Processor -- start 2024-09-13T23:43:31.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa334c20f40 1 -- start start 2024-09-13T23:43:31.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa334c20f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa334c20f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f300f92500 con 0x55f300f05000 2024-09-13T23:43:31.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa3333d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa3333d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52072/0 (socket says 172.21.15.167:52072) 2024-09-13T23:43:31.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa3333d9700 1 -- 172.21.15.167:0/2531627367 learned_addr learned my addr 172.21.15.167:0/2531627367 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa3333d9700 1 -- 172.21.15.167:0/2531627367 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f300f17ba0 con 0x55f300f05000 2024-09-13T23:43:31.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa3333d9700 1 --2- 172.21.15.167:0/2531627367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x55f301ccfe00 tx=0x55f301009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e763cffcf80b4f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa331bd6700 1 -- 172.21.15.167:0/2531627367 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f300fa9760 con 0x55f300f05000 2024-09-13T23:43:31.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa331bd6700 1 -- 172.21.15.167:0/2531627367 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f300fc9980 con 0x55f300f05000 2024-09-13T23:43:31.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.307+0000 7fa331bd6700 1 -- 172.21.15.167:0/2531627367 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f300fa94a0 con 0x55f300f05000 2024-09-13T23:43:31.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 -- 172.21.15.167:0/2531627367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 msgr2=0x55f3011ea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 --2- 172.21.15.167:0/2531627367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x55f301ccfe00 tx=0x55f301009800 comp rx=0 tx=0).stop 2024-09-13T23:43:31.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 -- 172.21.15.167:0/2531627367 shutdown_connections 2024-09-13T23:43:31.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 --2- 172.21.15.167:0/2531627367 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 -- 172.21.15.167:0/2531627367 >> 172.21.15.167:0/2531627367 conn(0x55f300f04c00 msgr2=0x55f301005000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 -- 172.21.15.167:0/2531627367 shutdown_connections 2024-09-13T23:43:31.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 -- 172.21.15.167:0/2531627367 wait complete. 2024-09-13T23:43:31.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 Processor -- start 2024-09-13T23:43:31.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 -- start start 2024-09-13T23:43:31.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f300f923c0 con 0x55f300f05000 2024-09-13T23:43:31.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa3333d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa3333d9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52082/0 (socket says 172.21.15.167:52082) 2024-09-13T23:43:31.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa3333d9700 1 -- 172.21.15.167:0/1736429486 learned_addr learned my addr 172.21.15.167:0/1736429486 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa3333d9700 1 -- 172.21.15.167:0/1736429486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f300f17a00 con 0x55f300f05000 2024-09-13T23:43:31.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa3333d9700 1 --2- 172.21.15.167:0/1736429486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55f301ccfef0 tx=0x55f301009980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa3303d3700 1 -- 172.21.15.167:0/1736429486 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f300fa9600 con 0x55f300f05000 2024-09-13T23:43:31.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa3303d3700 1 -- 172.21.15.167:0/1736429486 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f300fc9b00 con 0x55f300f05000 2024-09-13T23:43:31.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 -- 172.21.15.167:0/1736429486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f300f17860 con 0x55f300f05000 2024-09-13T23:43:31.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa3303d3700 1 -- 172.21.15.167:0/1736429486 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f300fa98c0 con 0x55f300f05000 2024-09-13T23:43:31.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.311+0000 7fa334c20f40 1 -- 172.21.15.167:0/1736429486 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f300f176c0 con 0x55f300f05000 2024-09-13T23:43:31.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa334c20f40 1 Processor -- start 2024-09-13T23:43:31.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa334c20f40 1 -- start start 2024-09-13T23:43:31.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa334c20f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 0x55f3011eab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa334c20f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f300f2a640 con 0x55f300f05800 2024-09-13T23:43:31.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa3303d3700 1 -- 172.21.15.167:0/1736429486 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f300ffeb00 con 0x55f300f05000 2024-09-13T23:43:31.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa333bda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 0x55f3011eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa333bda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 0x55f3011eab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52084/0 (socket says 172.21.15.167:52084) 2024-09-13T23:43:31.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa333bda700 1 -- 172.21.15.167:0/1989272937 learned_addr learned my addr 172.21.15.167:0/1989272937 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa3303d3700 1 --2- 172.21.15.167:0/1736429486 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3012b2000 0x55f3011eb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa3303d3700 1 -- 172.21.15.167:0/1736429486 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f300f6b180 con 0x55f300f05000 2024-09-13T23:43:31.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa3333d9700 1 --2- 172.21.15.167:0/1736429486 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3012b2000 0x55f3011eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa333bda700 1 -- 172.21.15.167:0/1989272937 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f300f16000 con 0x55f300f05800 2024-09-13T23:43:31.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa333bda700 1 --2- 172.21.15.167:0/1989272937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 0x55f3011eab00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55f3012922a0 tx=0x55f301276b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6a252c51b4e430 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa3333d9700 1 --2- 172.21.15.167:0/1736429486 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3012b2000 0x55f3011eb080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55f30121f590 tx=0x55f301276c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa32dbce700 1 -- 172.21.15.167:0/1989272937 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f300fa9600 con 0x55f300f05800 2024-09-13T23:43:31.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa32dbce700 1 -- 172.21.15.167:0/1989272937 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f300fc9b00 con 0x55f300f05800 2024-09-13T23:43:31.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa32dbce700 1 -- 172.21.15.167:0/1989272937 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f300fa98c0 con 0x55f300f05800 2024-09-13T23:43:31.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa334c20f40 1 -- 172.21.15.167:0/1989272937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 msgr2=0x55f3011eab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.315+0000 7fa334c20f40 1 --2- 172.21.15.167:0/1989272937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 0x55f3011eab00 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55f3012922a0 tx=0x55f301276b40 comp rx=0 tx=0).stop 2024-09-13T23:43:31.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa334c20f40 1 -- 172.21.15.167:0/1989272937 shutdown_connections 2024-09-13T23:43:31.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa334c20f40 1 --2- 172.21.15.167:0/1989272937 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 0x55f3011eab00 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa334c20f40 1 -- 172.21.15.167:0/1989272937 >> 172.21.15.167:0/1989272937 conn(0x55f300f05400 msgr2=0x55f301005800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa334c20f40 1 -- 172.21.15.167:0/1989272937 shutdown_connections 2024-09-13T23:43:31.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa334c20f40 1 -- 172.21.15.167:0/1989272937 wait complete. 2024-09-13T23:43:31.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa334c20f40 1 Processor -- start 2024-09-13T23:43:31.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa334c20f40 1 -- start start 2024-09-13T23:43:31.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa334c20f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 0x55f3011eab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa334c20f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f300f2a8c0 con 0x55f300f05800 2024-09-13T23:43:31.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa333bda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 0x55f3011eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa333bda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 0x55f3011eab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52092/0 (socket says 172.21.15.167:52092) 2024-09-13T23:43:31.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 learned_addr learned my addr 172.21.15.167:0/2066122154 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f300f16ea0 con 0x55f300f05800 2024-09-13T23:43:31.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa333bda700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 0x55f3011eab00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55f30121e420 tx=0x55f3012765a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa32c3cb700 1 -- 172.21.15.167:0/2066122154 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f300fa9760 con 0x55f300f05800 2024-09-13T23:43:31.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa32c3cb700 1 -- 172.21.15.167:0/2066122154 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f3012d2000 con 0x55f300f05800 2024-09-13T23:43:31.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa32c3cb700 1 -- 172.21.15.167:0/2066122154 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f300fa9600 con 0x55f300f05800 2024-09-13T23:43:31.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f300f16820 con 0x55f300f05800 2024-09-13T23:43:31.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.319+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f300f169c0 con 0x55f300f05800 2024-09-13T23:43:31.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa32c3cb700 1 -- 172.21.15.167:0/2066122154 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f300ffedc0 con 0x55f300f05800 2024-09-13T23:43:31.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa32c3cb700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3012b2c00 0x55f3011eb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa32c3cb700 1 -- 172.21.15.167:0/2066122154 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f300f6b340 con 0x55f300f05800 2024-09-13T23:43:31.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa332bd8700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3012b2c00 0x55f3011eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f3012b3000 0x55f3011ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3400 con 0x55f3012b3000 2024-09-13T23:43:31.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa333bda700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f3012b3000 0x55f3011ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa332bd8700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3012b2c00 0x55f3011eb600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55f301cce030 tx=0x55f30135c0c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa333bda700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f3012b3000 0x55f3011ebb80 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 20 realm 2024-09-13T23:43:31.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3000 2024-09-13T23:43:31.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:31.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3c00 con 0x55f3012b3000 2024-09-13T23:43:31.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f3012b3400 0x55f3011ec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3400 2024-09-13T23:43:31.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.323+0000 7fa3333d9700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f3012b3400 0x55f3011ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa3333d9700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f3012b3400 0x55f3011ec100 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa334c20f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:31.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3c00 con 0x55f3012b3400 2024-09-13T23:43:31.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa334c20f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:31.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa334c20f40 20 searching for the correct realm 2024-09-13T23:43:31.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa334c20f40 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f30136c000 0x55f3011ec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136c400 con 0x55f30136c000 2024-09-13T23:43:31.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa332bd8700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f30136c000 0x55f3011ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa332bd8700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f30136c000 0x55f3011ec680 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.327+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136c400 con 0x55f3012b3400 2024-09-13T23:43:31.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa3333d9700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f30136c400 0x55f3011ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c400 2024-09-13T23:43:31.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa333bda700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f30136c400 0x55f3011ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa333bda700 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f30136c400 0x55f3011ecc00 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3000 2024-09-13T23:43:31.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3000 2024-09-13T23:43:31.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c000 2024-09-13T23:43:31.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3000 2024-09-13T23:43:31.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.331+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3400 2024-09-13T23:43:31.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.335+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.335+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3400 2024-09-13T23:43:31.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.335+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.335+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3000 2024-09-13T23:43:31.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.335+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.335+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c400 2024-09-13T23:43:31.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.335+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.335+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3000 2024-09-13T23:43:31.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.335+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.335+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c400 2024-09-13T23:43:31.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.335+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.335+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3400 2024-09-13T23:43:31.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c000 2024-09-13T23:43:31.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3000 2024-09-13T23:43:31.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3400 2024-09-13T23:43:31.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c000 2024-09-13T23:43:31.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3000 2024-09-13T23:43:31.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3000 2024-09-13T23:43:31.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.339+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3000 2024-09-13T23:43:31.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c000 2024-09-13T23:43:31.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c400 2024-09-13T23:43:31.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3000 2024-09-13T23:43:31.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c400 2024-09-13T23:43:31.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c000 2024-09-13T23:43:31.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c400 2024-09-13T23:43:31.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.343+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3400 2024-09-13T23:43:31.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3400 2024-09-13T23:43:31.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c000 2024-09-13T23:43:31.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3400 2024-09-13T23:43:31.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f3012b3400 2024-09-13T23:43:31.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa334c20f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:31.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa334c20f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:31.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa334c20f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:31.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa334c20f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:31.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.347+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136c800 con 0x55f3012b3000 2024-09-13T23:43:31.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136cc00 con 0x55f3012b3400 2024-09-13T23:43:31.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:31.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136d000 con 0x55f3012b3400 2024-09-13T23:43:31.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:31.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 20 zone default found 2024-09-13T23:43:31.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 4 Realm: () 2024-09-13T23:43:31.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:31.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:31.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:31.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 20 zonegroup default 2024-09-13T23:43:31.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136d400 con 0x55f30136c400 2024-09-13T23:43:31.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136d800 con 0x55f30136c000 2024-09-13T23:43:31.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136dc00 con 0x55f30136c000 2024-09-13T23:43:31.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.351+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.355+0000 7fa334c20f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.355+0000 7fa334c20f40 20 started sync module instance, tier type = 2024-09-13T23:43:31.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.355+0000 7fa334c20f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:31.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.355+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136d800 con 0x55f3012b3400 2024-09-13T23:43:31.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.355+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.355+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94502184164096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136dc00 con 0x55f3012b3400 2024-09-13T23:43:31.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.355+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136cc00 con 0x55f3012b3000 2024-09-13T23:43:31.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.359+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94502184164096] v33'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.359+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.359+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94502184165248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136d000 con 0x55f3012b3000 2024-09-13T23:43:31.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.359+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136d800 con 0x55f3012b3400 2024-09-13T23:43:31.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.363+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94502184165248] v33'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.363+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.363+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94502184166400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3c00 con 0x55f3012b3400 2024-09-13T23:43:31.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.363+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c400 2024-09-13T23:43:31.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.363+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94502184166400] v33'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.367+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.367+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94502184167552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136c800 con 0x55f30136c400 2024-09-13T23:43:31.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.367+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136d400 con 0x55f3012b3400 2024-09-13T23:43:31.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.367+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94502184167552] v33'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.367+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.367+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94502184168704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136cc00 con 0x55f3012b3400 2024-09-13T23:43:31.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.367+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136d000 con 0x55f30136c000 2024-09-13T23:43:31.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.371+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.371+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94502184169856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301376000 con 0x55f30136c000 2024-09-13T23:43:31.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.371+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301376400 con 0x55f30136c000 2024-09-13T23:43:31.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.371+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94502184168704] v33'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94502184169856] v33'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94502184171008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301376800 con 0x55f30136c000 2024-09-13T23:43:31.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301376c00 con 0x55f30136c400 2024-09-13T23:43:31.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'105 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94502184172160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301377000 con 0x55f30136c400 2024-09-13T23:43:31.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa334c20f40 20 add_watcher() i=0 2024-09-13T23:43:31.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa334c20f40 20 add_watcher() i=1 2024-09-13T23:43:31.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa334c20f40 20 add_watcher() i=2 2024-09-13T23:43:31.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa334c20f40 20 add_watcher() i=3 2024-09-13T23:43:31.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa334c20f40 20 add_watcher() i=4 2024-09-13T23:43:31.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa334c20f40 20 add_watcher() i=5 2024-09-13T23:43:31.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94502184171008] v33'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.375+0000 7fa334c20f40 20 add_watcher() i=6 2024-09-13T23:43:31.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.379+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94502184172160] v33'106 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.379+0000 7fa334c20f40 20 add_watcher() i=7 2024-09-13T23:43:31.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.379+0000 7fa334c20f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:31.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.379+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301377400 con 0x55f3012b3000 2024-09-13T23:43:31.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.379+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.379+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94502185844736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301377800 con 0x55f3012b3000 2024-09-13T23:43:31.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94502185844736] v33'104 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa31a3a7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 1 Processor -- start 2024-09-13T23:43:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 1 -- start start 2024-09-13T23:43:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 0x55f3011ed180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f300f923c0 con 0x55f301376c00 2024-09-13T23:43:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa332bd8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 0x55f3011ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa332bd8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 0x55f3011ed180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52108/0 (socket says 172.21.15.167:52108) 2024-09-13T23:43:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa332bd8700 1 -- 172.21.15.167:0/165626793 learned_addr learned my addr 172.21.15.167:0/165626793 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa332bd8700 1 -- 172.21.15.167:0/165626793 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f300f16b60 con 0x55f301376c00 2024-09-13T23:43:31.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa332bd8700 1 --2- 172.21.15.167:0/165626793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 0x55f3011ed180 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55f3013860c0 tx=0x55f30135c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f56230ea2d837ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa319ba6700 1 -- 172.21.15.167:0/165626793 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f30138a000 con 0x55f301376c00 2024-09-13T23:43:31.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa319ba6700 1 -- 172.21.15.167:0/165626793 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f3012d2480 con 0x55f301376c00 2024-09-13T23:43:31.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa319ba6700 1 -- 172.21.15.167:0/165626793 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f30138a160 con 0x55f301376c00 2024-09-13T23:43:31.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 1 -- 172.21.15.167:0/165626793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 msgr2=0x55f3011ed180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 1 --2- 172.21.15.167:0/165626793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 0x55f3011ed180 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x55f3013860c0 tx=0x55f30135c060 comp rx=0 tx=0).stop 2024-09-13T23:43:31.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 1 -- 172.21.15.167:0/165626793 shutdown_connections 2024-09-13T23:43:31.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 1 --2- 172.21.15.167:0/165626793 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 0x55f3011ed180 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 1 -- 172.21.15.167:0/165626793 >> 172.21.15.167:0/165626793 conn(0x55f301377c00 msgr2=0x55f301006000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 1 -- 172.21.15.167:0/165626793 shutdown_connections 2024-09-13T23:43:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.383+0000 7fa334c20f40 1 -- 172.21.15.167:0/165626793 wait complete. 2024-09-13T23:43:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa334c20f40 1 Processor -- start 2024-09-13T23:43:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa334c20f40 1 -- start start 2024-09-13T23:43:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa334c20f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 0x55f3011ed180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa334c20f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f300f92500 con 0x55f301376c00 2024-09-13T23:43:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa332bd8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 0x55f3011ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa332bd8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 0x55f3011ed180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52120/0 (socket says 172.21.15.167:52120) 2024-09-13T23:43:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa332bd8700 1 -- 172.21.15.167:0/1573765903 learned_addr learned my addr 172.21.15.167:0/1573765903 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa332bd8700 1 -- 172.21.15.167:0/1573765903 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f300f16d00 con 0x55f301376c00 2024-09-13T23:43:31.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa332bd8700 1 --2- 172.21.15.167:0/1573765903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 0x55f3011ed180 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55f301387380 tx=0x55f30135c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa3183a3700 1 -- 172.21.15.167:0/1573765903 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f30138a2c0 con 0x55f301376c00 2024-09-13T23:43:31.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa3183a3700 1 -- 172.21.15.167:0/1573765903 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f3012d2600 con 0x55f301376c00 2024-09-13T23:43:31.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa3183a3700 1 -- 172.21.15.167:0/1573765903 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f30138a000 con 0x55f301376c00 2024-09-13T23:43:31.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa334c20f40 1 -- 172.21.15.167:0/1573765903 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f300f16340 con 0x55f301376c00 2024-09-13T23:43:31.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa334c20f40 1 -- 172.21.15.167:0/1573765903 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f300f16b60 con 0x55f301376c00 2024-09-13T23:43:31.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.387+0000 7fa3183a3700 1 -- 172.21.15.167:0/1573765903 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f300fffb80 con 0x55f301376c00 2024-09-13T23:43:31.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa3183a3700 1 --2- 172.21.15.167:0/1573765903 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f301402000 0x55f301408000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa3183a3700 1 -- 172.21.15.167:0/1573765903 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55f300f6b500 con 0x55f301376c00 2024-09-13T23:43:31.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 5 note: GC not initialized 2024-09-13T23:43:31.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa333bda700 1 --2- 172.21.15.167:0/1573765903 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f301402000 0x55f301408000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa333bda700 1 --2- 172.21.15.167:0/1573765903 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f301402000 0x55f301408000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55f30130f800 tx=0x55f301277500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa315b9e700 20 reqs_thread_entry: start 2024-09-13T23:43:31.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:31.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:31.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa313b9a700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:01 2024 2024-09-13T23:43:31.402 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:31.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa313378700 20 reqs_thread_entry: start 2024-09-13T23:43:31.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:31.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301376000 con 0x55f3012b3000 2024-09-13T23:43:31.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:31.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:31.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:31.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:31.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-13T23:43:31.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.391+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301376400 con 0x55f30136c400 2024-09-13T23:43:31.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:31.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 : miss 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136c800 con 0x55f30136c000 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(64 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-09-13T23:43:31.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136d000 con 0x55f30136c400 2024-09-13T23:43:31.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(65 foo.buckets [call] v33'17 uv17 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-09-13T23:43:31.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.395+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c400 2024-09-13T23:43:31.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 233+0+288 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa334c20f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:31.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa334c20f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-09-13T23:43:31.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa334c20f40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:31.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa334c20f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:31.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301402800 con 0x55f30136c400 2024-09-13T23:43:31.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v33'18 uv18 ondisk = 0) v8 ==== 317+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa334c20f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-09-13T23:43:31.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa334c20f40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:31.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa334c20f40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-13T23:43:31.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94502185847040 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301402c00 con 0x55f30136c000 2024-09-13T23:43:31.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== watch-notify(notify (1) cookie 94502184171008 notify 141733920769 ret 0) v3 ==== 380+0+0 (crc 0 0 0) 0x55f3013ced00 con 0x55f30136c000 2024-09-13T23:43:31.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa32dbce700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920769 cookie 94502184171008 notifier 4510 bl.length()=338 2024-09-13T23:43:31.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa32dbce700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-09-13T23:43:31.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa32dbce700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:31.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(68 notify.6 [notify cookie 94502185847040 out=8b] v0'0 uv105 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.399+0000 7fa32dbce700 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301377800 con 0x55f30136c000 2024-09-13T23:43:31.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.403+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== watch-notify(notify_complete (2) cookie 94502185847040 notify 141733920769 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55f3013ceea0 con 0x55f30136c000 2024-09-13T23:43:31.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.403+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.403+0000 7fa31a3a7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:31.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.403+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94502185844736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301403000 con 0x55f3012b3000 2024-09-13T23:43:31.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.407+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94502185844736] v33'105 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.407+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94502184164096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301403400 con 0x55f3012b3400 2024-09-13T23:43:31.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.407+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94502184164096] v33'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.407+0000 7fa334c20f40 20 remove_watcher() i=0 2024-09-13T23:43:31.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.407+0000 7fa334c20f40 2 removed watcher, disabling cache 2024-09-13T23:43:31.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.407+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94502184165248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301403800 con 0x55f3012b3000 2024-09-13T23:43:31.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.411+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94502184165248] v33'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f3012b3000 2024-09-13T23:43:31.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.411+0000 7fa334c20f40 20 remove_watcher() i=1 2024-09-13T23:43:31.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.411+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94502184166400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301403c00 con 0x55f3012b3400 2024-09-13T23:43:31.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.411+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94502184166400] v33'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.411+0000 7fa334c20f40 20 remove_watcher() i=2 2024-09-13T23:43:31.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.411+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94502184167552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301403400 con 0x55f30136c400 2024-09-13T23:43:31.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.415+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94502184167552] v33'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.415+0000 7fa334c20f40 20 remove_watcher() i=3 2024-09-13T23:43:31.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.415+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94502184168704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f301402800 con 0x55f3012b3400 2024-09-13T23:43:31.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.415+0000 7fa3333d9700 1 -- 172.21.15.167:0/2066122154 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94502184168704] v33'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353200 con 0x55f3012b3400 2024-09-13T23:43:31.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.415+0000 7fa334c20f40 20 remove_watcher() i=4 2024-09-13T23:43:31.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.415+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94502184169856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3c00 con 0x55f30136c000 2024-09-13T23:43:31.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.419+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94502184169856] v33'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.419+0000 7fa334c20f40 20 remove_watcher() i=5 2024-09-13T23:43:31.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.419+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94502184171008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f3012b3800 con 0x55f30136c000 2024-09-13T23:43:31.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94502184171008] v33'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301353440 con 0x55f30136c000 2024-09-13T23:43:31.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 20 remove_watcher() i=6 2024-09-13T23:43:31.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94502184172160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55f30136dc00 con 0x55f30136c400 2024-09-13T23:43:31.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa333bda700 1 -- 172.21.15.167:0/2066122154 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94502184172160] v33'107 uv105 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f301352b40 con 0x55f30136c400 2024-09-13T23:43:31.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 20 remove_watcher() i=7 2024-09-13T23:43:31.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f3012b3400 msgr2=0x55f3011ec100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f3012b3400 0x55f3011ec100 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f3012b3000 msgr2=0x55f3011ebb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f3012b3000 0x55f3011ebb80 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f30136c400 msgr2=0x55f3011ecc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f30136c400 0x55f3011ecc00 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f30136c000 msgr2=0x55f3011ec680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f30136c000 0x55f3011ec680 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3012b2c00 msgr2=0x55f3011eb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3012b2c00 0x55f3011eb600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55f301cce030 tx=0x55f30135c0c0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 msgr2=0x55f3011eab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa332bd8700 1 -- 172.21.15.167:0/2066122154 reap_dead start 2024-09-13T23:43:31.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 0x55f3011eab00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x55f30121e420 tx=0x55f3012765a0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 shutdown_connections 2024-09-13T23:43:31.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 --2- 172.21.15.167:0/2066122154 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05800 0x55f3011eab00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 >> 172.21.15.167:0/2066122154 conn(0x55f300f05400 msgr2=0x55f301005800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 shutdown_connections 2024-09-13T23:43:31.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 -- 172.21.15.167:0/2066122154 wait complete. 2024-09-13T23:43:31.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 -- 172.21.15.167:0/1573765903 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f301402000 msgr2=0x55f301408000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.423+0000 7fa334c20f40 1 --2- 172.21.15.167:0/1573765903 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f301402000 0x55f301408000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55f30130f800 tx=0x55f301277500 comp rx=0 tx=0).stop 2024-09-13T23:43:31.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 -- 172.21.15.167:0/1573765903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 msgr2=0x55f3011ed180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 --2- 172.21.15.167:0/1573765903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 0x55f3011ed180 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x55f301387380 tx=0x55f30135c180 comp rx=0 tx=0).stop 2024-09-13T23:43:31.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 -- 172.21.15.167:0/1573765903 shutdown_connections 2024-09-13T23:43:31.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 --2- 172.21.15.167:0/1573765903 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f301402000 0x55f301408000 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 --2- 172.21.15.167:0/1573765903 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f301376c00 0x55f3011ed180 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 -- 172.21.15.167:0/1573765903 >> 172.21.15.167:0/1573765903 conn(0x55f301377c00 msgr2=0x55f301006000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 -- 172.21.15.167:0/1573765903 shutdown_connections 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 -- 172.21.15.167:0/1573765903 wait complete. 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 -- 172.21.15.167:0/1736429486 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3012b2000 msgr2=0x55f3011eb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 --2- 172.21.15.167:0/1736429486 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3012b2000 0x55f3011eb080 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x55f30121f590 tx=0x55f301276c60 comp rx=0 tx=0).stop 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 -- 172.21.15.167:0/1736429486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 msgr2=0x55f3011ea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 --2- 172.21.15.167:0/1736429486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55f301ccfef0 tx=0x55f301009980 comp rx=0 tx=0).stop 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 -- 172.21.15.167:0/1736429486 shutdown_connections 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 --2- 172.21.15.167:0/1736429486 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3012b2000 0x55f3011eb080 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 --2- 172.21.15.167:0/1736429486 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f300f05000 0x55f3011ea580 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 -- 172.21.15.167:0/1736429486 >> 172.21.15.167:0/1736429486 conn(0x55f300f04c00 msgr2=0x55f301005000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 -- 172.21.15.167:0/1736429486 shutdown_connections 2024-09-13T23:43:31.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.427+0000 7fa334c20f40 1 -- 172.21.15.167:0/1736429486 wait complete. 2024-09-13T23:43:31.441 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-09-13T23:43:31.441 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-09-13T23:43:31.441 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-09-13T23:43:31.531 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:31.531 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:31.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fc0018a3f40 1 Processor -- start 2024-09-13T23:43:31.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fc0018a3f40 1 -- start start 2024-09-13T23:43:31.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fc0018a3f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55a87b94d000 0x55a87bc32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fc0018a3f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a87ba11500 con 0x55a87b94d400 2024-09-13T23:43:31.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fc0018a3f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a87b9daa00 con 0x55a87b94d000 2024-09-13T23:43:31.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fc00005c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55a87b94d000 0x55a87bc32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fbfff85b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55a87b94d400 0x55a87ba4c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45476/0 (socket says 172.21.15.167:45476) 2024-09-13T23:43:31.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fbfff85b700 1 -- 172.21.15.167:0/1249543788 learned_addr learned my addr 172.21.15.167:0/1249543788 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:31.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fbfff05a700 1 -- 172.21.15.167:0/1249543788 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183312086 0 0) 0x55a87baa8000 con 0x55a87b94d400 2024-09-13T23:43:31.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fbfff05a700 1 -- 172.21.15.167:0/1249543788 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55a87ba11680 con 0x55a87b94d400 2024-09-13T23:43:31.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fc00005c700 1 -- 172.21.15.167:0/1249543788 >> v1:172.21.15.12:6789/0 conn(0x55a87b94d400 legacy=0x55a87ba4c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fc00005c700 1 -- 172.21.15.167:0/1249543788 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87b95fd40 con 0x55a87b94d000 2024-09-13T23:43:31.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.523+0000 7fc00005c700 1 --2- 172.21.15.167:0/1249543788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55a87bc2da70 tx=0x55a87ba51980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac204fb64a3767b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fbfff05a700 1 -- 172.21.15.167:0/1249543788 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87b9f14a0 con 0x55a87b94d000 2024-09-13T23:43:31.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fbfff05a700 1 -- 172.21.15.167:0/1249543788 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a87ba11680 con 0x55a87b94d000 2024-09-13T23:43:31.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fbfff05a700 1 -- 172.21.15.167:0/1249543788 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87b9f1600 con 0x55a87b94d000 2024-09-13T23:43:31.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1249543788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 msgr2=0x55a87bc32580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/1249543788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x55a87bc2da70 tx=0x55a87ba51980 comp rx=0 tx=0).stop 2024-09-13T23:43:31.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1249543788 shutdown_connections 2024-09-13T23:43:31.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/1249543788 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1249543788 >> 172.21.15.167:0/1249543788 conn(0x55a87b94cc00 msgr2=0x55a87ba4c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1249543788 shutdown_connections 2024-09-13T23:43:31.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1249543788 wait complete. 2024-09-13T23:43:31.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc0018a3f40 1 Processor -- start 2024-09-13T23:43:31.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc0018a3f40 1 -- start start 2024-09-13T23:43:31.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc0018a3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc0018a3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87b9da500 con 0x55a87b94d000 2024-09-13T23:43:31.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc00005c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc00005c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52128/0 (socket says 172.21.15.167:52128) 2024-09-13T23:43:31.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc00005c700 1 -- 172.21.15.167:0/4284037379 learned_addr learned my addr 172.21.15.167:0/4284037379 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc00005c700 1 -- 172.21.15.167:0/4284037379 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87b95fba0 con 0x55a87b94d000 2024-09-13T23:43:31.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fc00005c700 1 --2- 172.21.15.167:0/4284037379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55a87c721e00 tx=0x55a87ba51800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882336fedf547840 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fbffe859700 1 -- 172.21.15.167:0/4284037379 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87b9f1760 con 0x55a87b94d000 2024-09-13T23:43:31.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fbffe859700 1 -- 172.21.15.167:0/4284037379 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a87ba11980 con 0x55a87b94d000 2024-09-13T23:43:31.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.527+0000 7fbffe859700 1 -- 172.21.15.167:0/4284037379 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87b9f11e0 con 0x55a87b94d000 2024-09-13T23:43:31.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4284037379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 msgr2=0x55a87bc32580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.539 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/4284037379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55a87c721e00 tx=0x55a87ba51800 comp rx=0 tx=0).stop 2024-09-13T23:43:31.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4284037379 shutdown_connections 2024-09-13T23:43:31.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/4284037379 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4284037379 >> 172.21.15.167:0/4284037379 conn(0x55a87b94cc00 msgr2=0x55a87ba4d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4284037379 shutdown_connections 2024-09-13T23:43:31.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4284037379 wait complete. 2024-09-13T23:43:31.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 Processor -- start 2024-09-13T23:43:31.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 -- start start 2024-09-13T23:43:31.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87b9da3c0 con 0x55a87b94d000 2024-09-13T23:43:31.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc00005c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc00005c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52130/0 (socket says 172.21.15.167:52130) 2024-09-13T23:43:31.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc00005c700 1 -- 172.21.15.167:0/3839796779 learned_addr learned my addr 172.21.15.167:0/3839796779 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc00005c700 1 -- 172.21.15.167:0/3839796779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87b95fa00 con 0x55a87b94d000 2024-09-13T23:43:31.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc00005c700 1 --2- 172.21.15.167:0/3839796779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55a87c721ef0 tx=0x55a87ba51980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fbffd056700 1 -- 172.21.15.167:0/3839796779 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87b9f14a0 con 0x55a87b94d000 2024-09-13T23:43:31.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fbffd056700 1 -- 172.21.15.167:0/3839796779 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a87ba11b00 con 0x55a87b94d000 2024-09-13T23:43:31.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3839796779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a87b95f860 con 0x55a87b94d000 2024-09-13T23:43:31.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fbffd056700 1 -- 172.21.15.167:0/3839796779 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87b9f1600 con 0x55a87b94d000 2024-09-13T23:43:31.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.531+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3839796779 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a87b95f6c0 con 0x55a87b94d000 2024-09-13T23:43:31.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fc0018a3f40 1 Processor -- start 2024-09-13T23:43:31.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fc0018a3f40 1 -- start start 2024-09-13T23:43:31.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fbffd056700 1 -- 172.21.15.167:0/3839796779 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55a87ba46b00 con 0x55a87b94d000 2024-09-13T23:43:31.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fc0018a3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 0x55a87bc32b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fc0018a3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87b972640 con 0x55a87b94d800 2024-09-13T23:43:31.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fbffd056700 1 --2- 172.21.15.167:0/3839796779 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87bcfa000 0x55a87bc33080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fbffd056700 1 -- 172.21.15.167:0/3839796779 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a87b9b3180 con 0x55a87b94d000 2024-09-13T23:43:31.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fc00085d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 0x55a87bc32b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fc00005c700 1 --2- 172.21.15.167:0/3839796779 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87bcfa000 0x55a87bc33080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fc00085d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 0x55a87bc32b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52134/0 (socket says 172.21.15.167:52134) 2024-09-13T23:43:31.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fc00085d700 1 -- 172.21.15.167:0/103266599 learned_addr learned my addr 172.21.15.167:0/103266599 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fc00085d700 1 -- 172.21.15.167:0/103266599 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87b95e000 con 0x55a87b94d800 2024-09-13T23:43:31.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fc00005c700 1 --2- 172.21.15.167:0/3839796779 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87bcfa000 0x55a87bc33080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55a87bc6cf60 tx=0x55a87bcbeb40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fc00085d700 1 --2- 172.21.15.167:0/103266599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 0x55a87bc32b00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55a87bc6c390 tx=0x55a87bcbee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a90208d0e53454f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fbffa851700 1 -- 172.21.15.167:0/103266599 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87b9f14a0 con 0x55a87b94d800 2024-09-13T23:43:31.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fbffa851700 1 -- 172.21.15.167:0/103266599 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a87ba11b00 con 0x55a87b94d800 2024-09-13T23:43:31.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.535+0000 7fbffa851700 1 -- 172.21.15.167:0/103266599 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87b9f1600 con 0x55a87b94d800 2024-09-13T23:43:31.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 -- 172.21.15.167:0/103266599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 msgr2=0x55a87bc32b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/103266599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 0x55a87bc32b00 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x55a87bc6c390 tx=0x55a87bcbee40 comp rx=0 tx=0).stop 2024-09-13T23:43:31.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 -- 172.21.15.167:0/103266599 shutdown_connections 2024-09-13T23:43:31.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/103266599 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 0x55a87bc32b00 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 -- 172.21.15.167:0/103266599 >> 172.21.15.167:0/103266599 conn(0x55a87b94d400 msgr2=0x55a87ba4d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 -- 172.21.15.167:0/103266599 shutdown_connections 2024-09-13T23:43:31.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 -- 172.21.15.167:0/103266599 wait complete. 2024-09-13T23:43:31.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 Processor -- start 2024-09-13T23:43:31.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 -- start start 2024-09-13T23:43:31.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 0x55a87bc32b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87b9728c0 con 0x55a87b94d800 2024-09-13T23:43:31.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc00085d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 0x55a87bc32b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc00085d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 0x55a87bc32b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52140/0 (socket says 172.21.15.167:52140) 2024-09-13T23:43:31.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 learned_addr learned my addr 172.21.15.167:0/4266000129 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87b95eea0 con 0x55a87b94d800 2024-09-13T23:43:31.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc00085d700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 0x55a87bc32b00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55a87bc6c420 tx=0x55a87bcbea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fbff904e700 1 -- 172.21.15.167:0/4266000129 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87b9f1760 con 0x55a87b94d800 2024-09-13T23:43:31.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fbff904e700 1 -- 172.21.15.167:0/4266000129 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a87bd24000 con 0x55a87b94d800 2024-09-13T23:43:31.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a87b95e820 con 0x55a87b94d800 2024-09-13T23:43:31.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fbff904e700 1 -- 172.21.15.167:0/4266000129 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87b9f14a0 con 0x55a87b94d800 2024-09-13T23:43:31.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.539+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a87b95e9c0 con 0x55a87b94d800 2024-09-13T23:43:31.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fbff904e700 1 -- 172.21.15.167:0/4266000129 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55a87ba46dc0 con 0x55a87b94d800 2024-09-13T23:43:31.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fbff904e700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87bcfac00 0x55a87bc33600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fbff904e700 1 -- 172.21.15.167:0/4266000129 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a87b9b3340 con 0x55a87b94d800 2024-09-13T23:43:31.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fbfff85b700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87bcfac00 0x55a87bc33600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a87bcfb000 0x55a87bc33b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb400 con 0x55a87bcfb000 2024-09-13T23:43:31.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc00085d700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a87bcfb000 0x55a87bc33b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fbfff85b700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87bcfac00 0x55a87bc33600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55a87c720030 tx=0x55a87bda20c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc00085d700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a87bcfb000 0x55a87bc33b80 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 20 realm 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb000 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfbc00 con 0x55a87bcfb000 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a87bcfb400 0x55a87bc34100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.543+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb400 2024-09-13T23:43:31.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fc00005c700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a87bcfb400 0x55a87bc34100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fc00005c700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a87bcfb400 0x55a87bc34100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fc0018a3f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:31.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfbc00 con 0x55a87bcfb400 2024-09-13T23:43:31.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fc0018a3f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:31.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fc0018a3f40 20 searching for the correct realm 2024-09-13T23:43:31.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a87bdba000 0x55a87bc34680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdba400 con 0x55a87bdba000 2024-09-13T23:43:31.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fbfff85b700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a87bdba000 0x55a87bc34680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.547+0000 7fbfff85b700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a87bdba000 0x55a87bc34680 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdba400 con 0x55a87bcfb400 2024-09-13T23:43:31.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fc00005c700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a87bdba400 0x55a87bc34c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba400 2024-09-13T23:43:31.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fc00085d700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a87bdba400 0x55a87bc34c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fc00085d700 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a87bdba400 0x55a87bc34c00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb000 2024-09-13T23:43:31.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb000 2024-09-13T23:43:31.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.551+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba000 2024-09-13T23:43:31.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb000 2024-09-13T23:43:31.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb400 2024-09-13T23:43:31.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb400 2024-09-13T23:43:31.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb000 2024-09-13T23:43:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba400 2024-09-13T23:43:31.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb000 2024-09-13T23:43:31.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.555+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba400 2024-09-13T23:43:31.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb400 2024-09-13T23:43:31.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba000 2024-09-13T23:43:31.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb000 2024-09-13T23:43:31.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb400 2024-09-13T23:43:31.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba000 2024-09-13T23:43:31.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb000 2024-09-13T23:43:31.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.559+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb000 2024-09-13T23:43:31.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb000 2024-09-13T23:43:31.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba000 2024-09-13T23:43:31.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba400 2024-09-13T23:43:31.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb000 2024-09-13T23:43:31.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba400 2024-09-13T23:43:31.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba000 2024-09-13T23:43:31.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.563+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.567+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba400 2024-09-13T23:43:31.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.567+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.567+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb400 2024-09-13T23:43:31.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.567+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.567+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb400 2024-09-13T23:43:31.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.567+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.567+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba000 2024-09-13T23:43:31.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.567+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.567+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb400 2024-09-13T23:43:31.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.567+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.567+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb400 2024-09-13T23:43:31.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:31.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:31.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:31.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:31.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdba800 con 0x55a87bcfb000 2024-09-13T23:43:31.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbac00 con 0x55a87bcfb400 2024-09-13T23:43:31.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:31.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbb000 con 0x55a87bcfb400 2024-09-13T23:43:31.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:31.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 zone default found 2024-09-13T23:43:31.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 4 Realm: () 2024-09-13T23:43:31.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:31.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:31.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:31.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 zonegroup default 2024-09-13T23:43:31.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbb400 con 0x55a87bdba400 2024-09-13T23:43:31.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.571+0000 7fc0018a3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.575+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.575+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbb800 con 0x55a87bdba000 2024-09-13T23:43:31.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.575+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.575+0000 7fc0018a3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.575+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.575+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbbc00 con 0x55a87bdba000 2024-09-13T23:43:31.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.575+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.575+0000 7fc0018a3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.575+0000 7fc0018a3f40 20 started sync module instance, tier type = 2024-09-13T23:43:31.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.575+0000 7fc0018a3f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:31.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.575+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbb800 con 0x55a87bcfb400 2024-09-13T23:43:31.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.579+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.579+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94182119217920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbbc00 con 0x55a87bcfb400 2024-09-13T23:43:31.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.579+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbac00 con 0x55a87bcfb000 2024-09-13T23:43:31.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.579+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94182119217920] v33'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.579+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.579+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94182119219072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbb000 con 0x55a87bcfb000 2024-09-13T23:43:31.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.579+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbb800 con 0x55a87bcfb400 2024-09-13T23:43:31.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.583+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94182119219072] v33'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.583+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.583+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94182119220224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfbc00 con 0x55a87bcfb400 2024-09-13T23:43:31.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.583+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba400 2024-09-13T23:43:31.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.587+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94182119220224] v33'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.587+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.587+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94182119221376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdba800 con 0x55a87bdba400 2024-09-13T23:43:31.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.587+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbb400 con 0x55a87bcfb400 2024-09-13T23:43:31.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.591+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94182119221376] v33'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.591+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.591+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94182119222528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbac00 con 0x55a87bcfb400 2024-09-13T23:43:31.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.591+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbb000 con 0x55a87bdba000 2024-09-13T23:43:31.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.591+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.591+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94182119223680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdc4000 con 0x55a87bdba000 2024-09-13T23:43:31.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.591+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdc4400 con 0x55a87bdba000 2024-09-13T23:43:31.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.591+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94182119222528] v33'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94182119223680] v33'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94182119224832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdc4800 con 0x55a87bdba000 2024-09-13T23:43:31.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdc4c00 con 0x55a87bdba400 2024-09-13T23:43:31.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'108 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94182119224832] v33'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94182119225984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdc5000 con 0x55a87bdba400 2024-09-13T23:43:31.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fc0018a3f40 20 add_watcher() i=0 2024-09-13T23:43:31.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fc0018a3f40 20 add_watcher() i=1 2024-09-13T23:43:31.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fc0018a3f40 20 add_watcher() i=2 2024-09-13T23:43:31.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fc0018a3f40 20 add_watcher() i=3 2024-09-13T23:43:31.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fc0018a3f40 20 add_watcher() i=4 2024-09-13T23:43:31.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fc0018a3f40 20 add_watcher() i=5 2024-09-13T23:43:31.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.595+0000 7fc0018a3f40 20 add_watcher() i=6 2024-09-13T23:43:31.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.599+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94182119225984] v33'109 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.599+0000 7fc0018a3f40 20 add_watcher() i=7 2024-09-13T23:43:31.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.599+0000 7fc0018a3f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:31.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.599+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdc5400 con 0x55a87bcfb000 2024-09-13T23:43:31.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.599+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.599+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94182120923136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdc5800 con 0x55a87bcfb000 2024-09-13T23:43:31.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94182120923136] v33'106 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fbfe702a700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:31.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:31.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:31.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 1 Processor -- start 2024-09-13T23:43:31.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 1 -- start start 2024-09-13T23:43:31.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 0x55a87bc35180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87b9da3c0 con 0x55a87bdc4c00 2024-09-13T23:43:31.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fbfff85b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 0x55a87bc35180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fbfff85b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 0x55a87bc35180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52144/0 (socket says 172.21.15.167:52144) 2024-09-13T23:43:31.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fbfff85b700 1 -- 172.21.15.167:0/3533555005 learned_addr learned my addr 172.21.15.167:0/3533555005 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fbfff85b700 1 -- 172.21.15.167:0/3533555005 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87b95eb60 con 0x55a87bdc4c00 2024-09-13T23:43:31.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fbfff85b700 1 --2- 172.21.15.167:0/3533555005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 0x55a87bc35180 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55a87bdd20c0 tx=0x55a87bda2060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87534e2b3f16efc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fbfe6829700 1 -- 172.21.15.167:0/3533555005 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87bdd6000 con 0x55a87bdc4c00 2024-09-13T23:43:31.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fbfe6829700 1 -- 172.21.15.167:0/3533555005 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a87bd24480 con 0x55a87bdc4c00 2024-09-13T23:43:31.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fbfe6829700 1 -- 172.21.15.167:0/3533555005 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87bdd6160 con 0x55a87bdc4c00 2024-09-13T23:43:31.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3533555005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 msgr2=0x55a87bc35180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/3533555005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 0x55a87bc35180 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x55a87bdd20c0 tx=0x55a87bda2060 comp rx=0 tx=0).stop 2024-09-13T23:43:31.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3533555005 shutdown_connections 2024-09-13T23:43:31.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/3533555005 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 0x55a87bc35180 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3533555005 >> 172.21.15.167:0/3533555005 conn(0x55a87bdc5c00 msgr2=0x55a87ba4e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3533555005 shutdown_connections 2024-09-13T23:43:31.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.603+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3533555005 wait complete. 2024-09-13T23:43:31.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fc0018a3f40 1 Processor -- start 2024-09-13T23:43:31.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fc0018a3f40 1 -- start start 2024-09-13T23:43:31.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fc0018a3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 0x55a87bc35180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fc0018a3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a87b9da500 con 0x55a87bdc4c00 2024-09-13T23:43:31.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fbfff85b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 0x55a87bc35180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fbfff85b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 0x55a87bc35180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52146/0 (socket says 172.21.15.167:52146) 2024-09-13T23:43:31.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fbfff85b700 1 -- 172.21.15.167:0/1530763741 learned_addr learned my addr 172.21.15.167:0/1530763741 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fbfff85b700 1 -- 172.21.15.167:0/1530763741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a87b95ed00 con 0x55a87bdc4c00 2024-09-13T23:43:31.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fbfff85b700 1 --2- 172.21.15.167:0/1530763741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 0x55a87bc35180 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55a87bdd3380 tx=0x55a87bda2180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fbfe5026700 1 -- 172.21.15.167:0/1530763741 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87bdd62c0 con 0x55a87bdc4c00 2024-09-13T23:43:31.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fbfe5026700 1 -- 172.21.15.167:0/1530763741 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a87bd24600 con 0x55a87bdc4c00 2024-09-13T23:43:31.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1530763741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a87b95e340 con 0x55a87bdc4c00 2024-09-13T23:43:31.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fbfe5026700 1 -- 172.21.15.167:0/1530763741 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a87bdd6000 con 0x55a87bdc4c00 2024-09-13T23:43:31.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1530763741 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a87b95eb60 con 0x55a87bdc4c00 2024-09-13T23:43:31.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.607+0000 7fbfe5026700 1 -- 172.21.15.167:0/1530763741 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55a87ba47b80 con 0x55a87bdc4c00 2024-09-13T23:43:31.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fbfe5026700 1 --2- 172.21.15.167:0/1530763741 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87be54000 0x55a87be5a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fbfe5026700 1 -- 172.21.15.167:0/1530763741 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55a87b9b3500 con 0x55a87bdc4c00 2024-09-13T23:43:31.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc0018a3f40 5 note: GC not initialized 2024-09-13T23:43:31.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc00085d700 1 --2- 172.21.15.167:0/1530763741 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87be54000 0x55a87be5a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc00085d700 1 --2- 172.21.15.167:0/1530763741 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87be54000 0x55a87be5a000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55a87bd55800 tx=0x55a87bcbf500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fbfe2821700 20 reqs_thread_entry: start 2024-09-13T23:43:31.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc0018a3f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:31.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc0018a3f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:31.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fbfe081d700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:01 2024 2024-09-13T23:43:31.624 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:31.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fbfdfffb700 20 reqs_thread_entry: start 2024-09-13T23:43:31.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc0018a3f40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-13T23:43:31.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdc4000 con 0x55a87bdba400 2024-09-13T23:43:31.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc0018a3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc0018a3f40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-09-13T23:43:31.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc0018a3f40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:43:31.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc0018a3f40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-09-13T23:43:31.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc0018a3f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.611+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdc4400 con 0x55a87bdba000 2024-09-13T23:43:31.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fc0018a3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fc0018a3f40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-09-13T23:43:31.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fc0018a3f40 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-13T23:43:31.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fc0018a3f40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-13T23:43:31.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fc0018a3f40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-13T23:43:31.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdba800 con 0x55a87bdba400 2024-09-13T23:43:31.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~346] v33'1 uv1 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fc0018a3f40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-09-13T23:43:31.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fc0018a3f40 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:43:31.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fc0018a3f40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-09-13T23:43:31.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94182120925440 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbb000 con 0x55a87bdba000 2024-09-13T23:43:31.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== watch-notify(notify (1) cookie 94182119223680 notify 141733920770 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x55a87be1ed00 con 0x55a87bdba000 2024-09-13T23:43:31.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fbffa851700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920770 cookie 94182119223680 notifier 4518 bl.length()=522 2024-09-13T23:43:31.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fbffa851700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-09-13T23:43:31.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(65 notify.5 [notify cookie 94182120925440 out=8b] v0'0 uv108 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fbffa851700 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:43:31.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.615+0000 7fbffa851700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87be54400 con 0x55a87bdba000 2024-09-13T23:43:31.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== watch-notify(notify_complete (2) cookie 94182120925440 notify 141733920770 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a87be1eea0 con 0x55a87bdba000 2024-09-13T23:43:31.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bdba000 2024-09-13T23:43:31.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v33'3 uv3 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fc0018a3f40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-13T23:43:31.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fc0018a3f40 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-13T23:43:31.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fc0018a3f40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-13T23:43:31.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94182120925440 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87be54400 con 0x55a87bcfb400 2024-09-13T23:43:31.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94182119220224 notify 141733920786 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55a87bcb2680 con 0x55a87bcfb400 2024-09-13T23:43:31.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fbffa851700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920786 cookie 94182119220224 notifier 4518 bl.length()=177 2024-09-13T23:43:31.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fbffa851700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-13T23:43:31.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fbffa851700 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-13T23:43:31.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(68 notify.2 [notify cookie 94182120925440 out=8b] v0'0 uv108 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.619+0000 7fbffa851700 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87be54800 con 0x55a87bcfb400 2024-09-13T23:43:31.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.623+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== watch-notify(notify_complete (2) cookie 94182120925440 notify 141733920786 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55a87bcb2820 con 0x55a87bcfb400 2024-09-13T23:43:31.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.623+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.632 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:31.632 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "fud", 2024-09-13T23:43:31.632 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Fud", 2024-09-13T23:43:31.632 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "", 2024-09-13T23:43:31.632 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:31.632 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 1, 2024-09-13T23:43:31.632 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:31.632 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:31.632 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:31.632 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "fud", 2024-09-13T23:43:31.633 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-09-13T23:43:31.633 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-09-13T23:43:31.633 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:31.633 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:31.633 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:31.633 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:31.633 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:31.633 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:31.633 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:31.633 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:31.634 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:31.635 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:31.635 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:31.635 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:31.635 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:31.635 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:31.635 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.623+0000 7fbfe702a700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.623+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94182120923136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87be54c00 con 0x55a87bcfb000 2024-09-13T23:43:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.627+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94182120923136] v33'107 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.627+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94182119217920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87be55000 con 0x55a87bcfb400 2024-09-13T23:43:31.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.627+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94182119217920] v33'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.627+0000 7fc0018a3f40 20 remove_watcher() i=0 2024-09-13T23:43:31.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.627+0000 7fc0018a3f40 2 removed watcher, disabling cache 2024-09-13T23:43:31.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.627+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94182119219072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87be55400 con 0x55a87bcfb000 2024-09-13T23:43:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.631+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94182119219072] v33'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bcfb000 2024-09-13T23:43:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.631+0000 7fc0018a3f40 20 remove_watcher() i=1 2024-09-13T23:43:31.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.631+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94182119220224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87be55800 con 0x55a87bcfb400 2024-09-13T23:43:31.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.635+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94182119220224] v33'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.635+0000 7fc0018a3f40 20 remove_watcher() i=2 2024-09-13T23:43:31.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.635+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94182119221376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87be55c00 con 0x55a87bdba400 2024-09-13T23:43:31.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.635+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94182119221376] v33'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.635+0000 7fc0018a3f40 20 remove_watcher() i=3 2024-09-13T23:43:31.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.635+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94182119222528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bcfb800 con 0x55a87bcfb400 2024-09-13T23:43:31.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.639+0000 7fc00005c700 1 -- 172.21.15.167:0/4266000129 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94182119222528] v33'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7200 con 0x55a87bcfb400 2024-09-13T23:43:31.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.639+0000 7fc0018a3f40 20 remove_watcher() i=4 2024-09-13T23:43:31.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.639+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94182119223680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdbb000 con 0x55a87bdba000 2024-09-13T23:43:31.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.643+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94182119223680] v33'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.643+0000 7fc0018a3f40 20 remove_watcher() i=5 2024-09-13T23:43:31.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.643+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94182119224832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87bdc4400 con 0x55a87bdba000 2024-09-13T23:43:31.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.643+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94182119224832] v33'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb7440 con 0x55a87bdba000 2024-09-13T23:43:31.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.643+0000 7fc0018a3f40 20 remove_watcher() i=6 2024-09-13T23:43:31.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.643+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94182119225984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55a87be54c00 con 0x55a87bdba400 2024-09-13T23:43:31.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc00085d700 1 -- 172.21.15.167:0/4266000129 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94182119225984] v33'110 uv108 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55a87bdb6b40 con 0x55a87bdba400 2024-09-13T23:43:31.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 20 remove_watcher() i=7 2024-09-13T23:43:31.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a87bcfb400 msgr2=0x55a87bc34100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a87bcfb400 0x55a87bc34100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a87bcfb000 msgr2=0x55a87bc33b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a87bcfb000 0x55a87bc33b80 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a87bdba400 msgr2=0x55a87bc34c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a87bdba400 0x55a87bc34c00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a87bdba000 msgr2=0x55a87bc34680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a87bdba000 0x55a87bc34680 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87bcfac00 msgr2=0x55a87bc33600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87bcfac00 0x55a87bc33600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x55a87c720030 tx=0x55a87bda20c0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 msgr2=0x55a87bc32b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/4266000129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d800 0x55a87bc32b00 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x55a87bc6c420 tx=0x55a87bcbea80 comp rx=0 tx=0).stop 2024-09-13T23:43:31.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 reap_dead start 2024-09-13T23:43:31.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fbfff85b700 1 -- 172.21.15.167:0/4266000129 reap_dead start 2024-09-13T23:43:31.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 shutdown_connections 2024-09-13T23:43:31.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 >> 172.21.15.167:0/4266000129 conn(0x55a87b94d400 msgr2=0x55a87ba4d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 shutdown_connections 2024-09-13T23:43:31.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/4266000129 wait complete. 2024-09-13T23:43:31.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1530763741 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87be54000 msgr2=0x55a87be5a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/1530763741 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87be54000 0x55a87be5a000 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x55a87bd55800 tx=0x55a87bcbf500 comp rx=0 tx=0).stop 2024-09-13T23:43:31.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1530763741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 msgr2=0x55a87bc35180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/1530763741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 0x55a87bc35180 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55a87bdd3380 tx=0x55a87bda2180 comp rx=0 tx=0).stop 2024-09-13T23:43:31.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1530763741 shutdown_connections 2024-09-13T23:43:31.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/1530763741 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87be54000 0x55a87be5a000 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/1530763741 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87bdc4c00 0x55a87bc35180 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1530763741 >> 172.21.15.167:0/1530763741 conn(0x55a87bdc5c00 msgr2=0x55a87ba4e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1530763741 shutdown_connections 2024-09-13T23:43:31.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/1530763741 wait complete. 2024-09-13T23:43:31.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3839796779 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87bcfa000 msgr2=0x55a87bc33080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/3839796779 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87bcfa000 0x55a87bc33080 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x55a87bc6cf60 tx=0x55a87bcbeb40 comp rx=0 tx=0).stop 2024-09-13T23:43:31.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3839796779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 msgr2=0x55a87bc32580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/3839796779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x55a87c721ef0 tx=0x55a87ba51980 comp rx=0 tx=0).stop 2024-09-13T23:43:31.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3839796779 shutdown_connections 2024-09-13T23:43:31.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/3839796779 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a87bcfa000 0x55a87bc33080 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 --2- 172.21.15.167:0/3839796779 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a87b94d000 0x55a87bc32580 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.647+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3839796779 >> 172.21.15.167:0/3839796779 conn(0x55a87b94cc00 msgr2=0x55a87ba4d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.651+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3839796779 shutdown_connections 2024-09-13T23:43:31.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.651+0000 7fc0018a3f40 1 -- 172.21.15.167:0/3839796779 wait complete. 2024-09-13T23:43:31.664 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:31.674 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-09-13T23:43:31.682 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:31.682 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:myfoo'] 2024-09-13T23:43:31.682 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:myfoo'] 2024-09-13T23:43:31.682 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:myfoo 2024-09-13T23:43:31.719 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:31.719 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68f402f40 1 Processor -- start 2024-09-13T23:43:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68f402f40 1 -- start start 2024-09-13T23:43:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68f402f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55663bd2b000 0x55663c010580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68f402f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55663bdef500 con 0x55663bd2b400 2024-09-13T23:43:31.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68f402f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55663bdb8a00 con 0x55663bd2b000 2024-09-13T23:43:31.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68dbbb700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55663bd2b000 0x55663c010580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68d3ba700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55663bd2b400 0x55663be2a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45484/0 (socket says 172.21.15.167:45484) 2024-09-13T23:43:31.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1187402325 learned_addr learned my addr 172.21.15.167:0/1187402325 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:31.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68cbb9700 1 -- 172.21.15.167:0/1187402325 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154651573 0 0) 0x55663be86000 con 0x55663bd2b400 2024-09-13T23:43:31.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68cbb9700 1 -- 172.21.15.167:0/1187402325 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55663bdef680 con 0x55663bd2b400 2024-09-13T23:43:31.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1187402325 >> v1:172.21.15.12:6789/0 conn(0x55663bd2b400 legacy=0x55663be2a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1187402325 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55663bd3dd40 con 0x55663bd2b000 2024-09-13T23:43:31.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68dbbb700 1 --2- 172.21.15.167:0/1187402325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55663c00ba70 tx=0x55663be2f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6dd52f61dc0a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68cbb9700 1 -- 172.21.15.167:0/1187402325 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663bdcf4a0 con 0x55663bd2b000 2024-09-13T23:43:31.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68cbb9700 1 -- 172.21.15.167:0/1187402325 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55663bdef680 con 0x55663bd2b000 2024-09-13T23:43:31.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.711+0000 7fc68cbb9700 1 -- 172.21.15.167:0/1187402325 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663bdcf600 con 0x55663bd2b000 2024-09-13T23:43:31.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 -- 172.21.15.167:0/1187402325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 msgr2=0x55663c010580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 --2- 172.21.15.167:0/1187402325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x55663c00ba70 tx=0x55663be2f980 comp rx=0 tx=0).stop 2024-09-13T23:43:31.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 -- 172.21.15.167:0/1187402325 shutdown_connections 2024-09-13T23:43:31.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 --2- 172.21.15.167:0/1187402325 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 -- 172.21.15.167:0/1187402325 >> 172.21.15.167:0/1187402325 conn(0x55663bd2ac00 msgr2=0x55663be2a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 -- 172.21.15.167:0/1187402325 shutdown_connections 2024-09-13T23:43:31.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 -- 172.21.15.167:0/1187402325 wait complete. 2024-09-13T23:43:31.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 Processor -- start 2024-09-13T23:43:31.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 -- start start 2024-09-13T23:43:31.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663bdb8500 con 0x55663bd2b000 2024-09-13T23:43:31.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68dbbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68dbbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52168/0 (socket says 172.21.15.167:52168) 2024-09-13T23:43:31.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68dbbb700 1 -- 172.21.15.167:0/3818831626 learned_addr learned my addr 172.21.15.167:0/3818831626 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68dbbb700 1 -- 172.21.15.167:0/3818831626 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55663bd3dba0 con 0x55663bd2b000 2024-09-13T23:43:31.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68dbbb700 1 --2- 172.21.15.167:0/3818831626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55663caf5e00 tx=0x55663be2f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddcdb1c25f8529e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68c3b8700 1 -- 172.21.15.167:0/3818831626 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663bdcf760 con 0x55663bd2b000 2024-09-13T23:43:31.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68c3b8700 1 -- 172.21.15.167:0/3818831626 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55663bdef980 con 0x55663bd2b000 2024-09-13T23:43:31.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68c3b8700 1 -- 172.21.15.167:0/3818831626 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663bdcf4a0 con 0x55663bd2b000 2024-09-13T23:43:31.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 -- 172.21.15.167:0/3818831626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 msgr2=0x55663c010580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 --2- 172.21.15.167:0/3818831626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x55663caf5e00 tx=0x55663be2f800 comp rx=0 tx=0).stop 2024-09-13T23:43:31.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 -- 172.21.15.167:0/3818831626 shutdown_connections 2024-09-13T23:43:31.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 --2- 172.21.15.167:0/3818831626 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 -- 172.21.15.167:0/3818831626 >> 172.21.15.167:0/3818831626 conn(0x55663bd2ac00 msgr2=0x55663be2b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 -- 172.21.15.167:0/3818831626 shutdown_connections 2024-09-13T23:43:31.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.715+0000 7fc68f402f40 1 -- 172.21.15.167:0/3818831626 wait complete. 2024-09-13T23:43:31.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68f402f40 1 Processor -- start 2024-09-13T23:43:31.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68f402f40 1 -- start start 2024-09-13T23:43:31.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68f402f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68f402f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663bdb83c0 con 0x55663bd2b000 2024-09-13T23:43:31.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68dbbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68dbbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52178/0 (socket says 172.21.15.167:52178) 2024-09-13T23:43:31.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68dbbb700 1 -- 172.21.15.167:0/3995724225 learned_addr learned my addr 172.21.15.167:0/3995724225 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68dbbb700 1 -- 172.21.15.167:0/3995724225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55663bd3da00 con 0x55663bd2b000 2024-09-13T23:43:31.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68dbbb700 1 --2- 172.21.15.167:0/3995724225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x55663caf5ef0 tx=0x55663be2f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68abb5700 1 -- 172.21.15.167:0/3995724225 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663bdcf600 con 0x55663bd2b000 2024-09-13T23:43:31.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68abb5700 1 -- 172.21.15.167:0/3995724225 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55663bdefb00 con 0x55663bd2b000 2024-09-13T23:43:31.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68f402f40 1 -- 172.21.15.167:0/3995724225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55663bd3d860 con 0x55663bd2b000 2024-09-13T23:43:31.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68abb5700 1 -- 172.21.15.167:0/3995724225 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663bdcf8c0 con 0x55663bd2b000 2024-09-13T23:43:31.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68f402f40 1 -- 172.21.15.167:0/3995724225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55663bd3d6c0 con 0x55663bd2b000 2024-09-13T23:43:31.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68abb5700 1 -- 172.21.15.167:0/3995724225 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55663be24b00 con 0x55663bd2b000 2024-09-13T23:43:31.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68f402f40 1 Processor -- start 2024-09-13T23:43:31.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68abb5700 1 --2- 172.21.15.167:0/3995724225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c0ca000 0x55663c011080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68f402f40 1 -- start start 2024-09-13T23:43:31.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68abb5700 1 -- 172.21.15.167:0/3995724225 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55663bd91180 con 0x55663bd2b000 2024-09-13T23:43:31.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68e3bc700 1 --2- 172.21.15.167:0/3995724225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c0ca000 0x55663c011080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.719+0000 7fc68f402f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 0x55663c010b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68f402f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663bdb8000 con 0x55663bd2b800 2024-09-13T23:43:31.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68dbbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 0x55663c010b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68dbbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 0x55663c010b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52182/0 (socket says 172.21.15.167:52182) 2024-09-13T23:43:31.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68dbbb700 1 -- 172.21.15.167:0/310069133 learned_addr learned my addr 172.21.15.167:0/310069133 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68dbbb700 1 -- 172.21.15.167:0/310069133 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55663bd3c000 con 0x55663bd2b800 2024-09-13T23:43:31.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68e3bc700 1 --2- 172.21.15.167:0/3995724225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c0ca000 0x55663c011080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55663c0561b0 tx=0x55663c08eb40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68dbbb700 1 --2- 172.21.15.167:0/310069133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 0x55663c010b00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55663c057020 tx=0x55663c08ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99be299fc047517f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc6883b0700 1 -- 172.21.15.167:0/310069133 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663bdcf600 con 0x55663bd2b800 2024-09-13T23:43:31.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc6883b0700 1 -- 172.21.15.167:0/310069133 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55663bdefe00 con 0x55663bd2b800 2024-09-13T23:43:31.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc6883b0700 1 -- 172.21.15.167:0/310069133 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663bdcf8c0 con 0x55663bd2b800 2024-09-13T23:43:31.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68f402f40 1 -- 172.21.15.167:0/310069133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 msgr2=0x55663c010b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68f402f40 1 --2- 172.21.15.167:0/310069133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 0x55663c010b00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x55663c057020 tx=0x55663c08ec60 comp rx=0 tx=0).stop 2024-09-13T23:43:31.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68f402f40 1 -- 172.21.15.167:0/310069133 shutdown_connections 2024-09-13T23:43:31.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68f402f40 1 --2- 172.21.15.167:0/310069133 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 0x55663c010b00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68f402f40 1 -- 172.21.15.167:0/310069133 >> 172.21.15.167:0/310069133 conn(0x55663bd2b400 msgr2=0x55663be2b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68f402f40 1 -- 172.21.15.167:0/310069133 shutdown_connections 2024-09-13T23:43:31.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68f402f40 1 -- 172.21.15.167:0/310069133 wait complete. 2024-09-13T23:43:31.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68f402f40 1 Processor -- start 2024-09-13T23:43:31.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68f402f40 1 -- start start 2024-09-13T23:43:31.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.723+0000 7fc68f402f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 0x55663c010b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.727+0000 7fc68f402f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663bd50640 con 0x55663bd2b800 2024-09-13T23:43:31.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.727+0000 7fc68dbbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 0x55663c010b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.727+0000 7fc68dbbb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 0x55663c010b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52186/0 (socket says 172.21.15.167:52186) 2024-09-13T23:43:31.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.727+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 learned_addr learned my addr 172.21.15.167:0/1212737929 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.727+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55663bd3cea0 con 0x55663bd2b800 2024-09-13T23:43:31.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.727+0000 7fc68dbbb700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 0x55663c010b00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55663c0c4b40 tx=0x55663c08ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.727+0000 7fc686bad700 1 -- 172.21.15.167:0/1212737929 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663bdcf760 con 0x55663bd2b800 2024-09-13T23:43:31.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.727+0000 7fc686bad700 1 -- 172.21.15.167:0/1212737929 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55663bdefb00 con 0x55663bd2b800 2024-09-13T23:43:31.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.727+0000 7fc686bad700 1 -- 172.21.15.167:0/1212737929 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663bdcf4a0 con 0x55663bd2b800 2024-09-13T23:43:31.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.727+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55663bd3c820 con 0x55663bd2b800 2024-09-13T23:43:31.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.727+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55663bd3c9c0 con 0x55663bd2b800 2024-09-13T23:43:31.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.727+0000 7fc686bad700 1 -- 172.21.15.167:0/1212737929 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55663be24dc0 con 0x55663bd2b800 2024-09-13T23:43:31.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc686bad700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c0cb000 0x55663c011600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc686bad700 1 -- 172.21.15.167:0/1212737929 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55663bd91340 con 0x55663bd2b800 2024-09-13T23:43:31.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55663c0cb400 0x55663c011b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68d3ba700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c0cb000 0x55663c011600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c0cb800 con 0x55663c0cb400 2024-09-13T23:43:31.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68e3bc700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55663c0cb400 0x55663c011b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68d3ba700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c0cb000 0x55663c011600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55663caf4030 tx=0x55663c152000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68e3bc700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55663c0cb400 0x55663c011b80 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 20 realm 2024-09-13T23:43:31.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c0cbc00 con 0x55663c0cb400 2024-09-13T23:43:31.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:31.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c0cb800 con 0x55663c0cb400 2024-09-13T23:43:31.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55663c0cbc00 0x55663c012100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a000 con 0x55663c0cbc00 2024-09-13T23:43:31.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.731+0000 7fc68dbbb700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55663c0cbc00 0x55663c012100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68dbbb700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55663c0cbc00 0x55663c012100 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68f402f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:31.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68f402f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cbc00 2024-09-13T23:43:31.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68f402f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:31.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68f402f40 20 searching for the correct realm 2024-09-13T23:43:31.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68f402f40 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55663c15a800 0x55663c012680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15ac00 con 0x55663c15a800 2024-09-13T23:43:31.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68d3ba700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55663c15a800 0x55663c012680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68d3ba700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55663c15a800 0x55663c012680 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15ac00 con 0x55663c0cbc00 2024-09-13T23:43:31.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68dbbb700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55663c15a000 0x55663c012c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a000 2024-09-13T23:43:31.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.735+0000 7fc68e3bc700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55663c15a000 0x55663c012c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68e3bc700 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55663c15a000 0x55663c012c00 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cb400 2024-09-13T23:43:31.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cb400 2024-09-13T23:43:31.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a800 2024-09-13T23:43:31.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cb400 2024-09-13T23:43:31.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cbc00 2024-09-13T23:43:31.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.739+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cbc00 2024-09-13T23:43:31.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cb400 2024-09-13T23:43:31.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a000 2024-09-13T23:43:31.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cb400 2024-09-13T23:43:31.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a000 2024-09-13T23:43:31.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cbc00 2024-09-13T23:43:31.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a800 2024-09-13T23:43:31.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.743+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cb400 2024-09-13T23:43:31.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.747+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.747+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cbc00 2024-09-13T23:43:31.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.747+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.747+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a800 2024-09-13T23:43:31.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.747+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.747+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cb400 2024-09-13T23:43:31.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.747+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.747+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cb400 2024-09-13T23:43:31.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.747+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.747+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cb400 2024-09-13T23:43:31.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.747+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.747+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a800 2024-09-13T23:43:31.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a000 2024-09-13T23:43:31.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cb400 2024-09-13T23:43:31.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a000 2024-09-13T23:43:31.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a800 2024-09-13T23:43:31.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a000 2024-09-13T23:43:31.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cbc00 2024-09-13T23:43:31.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.751+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cbc00 2024-09-13T23:43:31.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a800 2024-09-13T23:43:31.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cbc00 2024-09-13T23:43:31.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cbc00 2024-09-13T23:43:31.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68f402f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:31.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68f402f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:31.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68f402f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:31.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68f402f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:31.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68f402f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15b000 con 0x55663c0cb400 2024-09-13T23:43:31.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68f402f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68f402f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.755+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15b400 con 0x55663c0cbc00 2024-09-13T23:43:31.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:31.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15b800 con 0x55663c0cbc00 2024-09-13T23:43:31.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:31.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 zone default found 2024-09-13T23:43:31.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 4 Realm: () 2024-09-13T23:43:31.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:31.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:31.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:31.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 zonegroup default 2024-09-13T23:43:31.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15bc00 con 0x55663c15a000 2024-09-13T23:43:31.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15b000 con 0x55663c15a800 2024-09-13T23:43:31.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15bc00 con 0x55663c15a800 2024-09-13T23:43:31.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 started sync module instance, tier type = 2024-09-13T23:43:31.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:31.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.759+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c0cb800 con 0x55663c0cbc00 2024-09-13T23:43:31.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.763+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.763+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93897581689600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15ac00 con 0x55663c0cbc00 2024-09-13T23:43:31.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.763+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c0cb400 2024-09-13T23:43:31.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.767+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.767+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93897581690752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15b400 con 0x55663c0cb400 2024-09-13T23:43:31.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.767+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15b800 con 0x55663c0cbc00 2024-09-13T23:43:31.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.767+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 93897581689600] v33'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.771+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93897581690752] v33'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.771+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.771+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93897581691904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15b000 con 0x55663c0cbc00 2024-09-13T23:43:31.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.771+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15bc00 con 0x55663c15a000 2024-09-13T23:43:31.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.771+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.771+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93897581693056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c0ca800 con 0x55663c15a000 2024-09-13T23:43:31.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.771+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c0cb800 con 0x55663c0cbc00 2024-09-13T23:43:31.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.771+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 93897581691904] v33'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.775+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 93897581693056] v33'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.775+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.775+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93897581694208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c0ca800 con 0x55663c0cbc00 2024-09-13T23:43:31.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.775+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a800 2024-09-13T23:43:31.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.779+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.779+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93897581695360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15b400 con 0x55663c15a800 2024-09-13T23:43:31.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.779+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15bc00 con 0x55663c15a800 2024-09-13T23:43:31.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.779+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93897581694208] v33'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.779+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93897581695360] v33'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.779+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.779+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93897581696512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15a400 con 0x55663c15a800 2024-09-13T23:43:31.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.779+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15b400 con 0x55663c15a000 2024-09-13T23:43:31.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'111 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93897581697664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c15bc00 con 0x55663c15a000 2024-09-13T23:43:31.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68f402f40 20 add_watcher() i=0 2024-09-13T23:43:31.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93897581696512] v33'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68f402f40 20 add_watcher() i=1 2024-09-13T23:43:31.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68f402f40 20 add_watcher() i=2 2024-09-13T23:43:31.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68f402f40 20 add_watcher() i=3 2024-09-13T23:43:31.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68f402f40 20 add_watcher() i=4 2024-09-13T23:43:31.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68f402f40 20 add_watcher() i=5 2024-09-13T23:43:31.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68f402f40 20 add_watcher() i=6 2024-09-13T23:43:31.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 93897581697664] v33'112 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68f402f40 20 add_watcher() i=7 2024-09-13T23:43:31.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68f402f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:31.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c16c000 con 0x55663c0cb400 2024-09-13T23:43:31.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.783+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93897583108096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c16c400 con 0x55663c0cb400 2024-09-13T23:43:31.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.787+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93897583108096] v33'108 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.787+0000 7fc674b89700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:31.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.787+0000 7fc68f402f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:31.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.787+0000 7fc68f402f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:31.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.787+0000 7fc68f402f40 1 Processor -- start 2024-09-13T23:43:31.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.787+0000 7fc68f402f40 1 -- start start 2024-09-13T23:43:31.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.787+0000 7fc68f402f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16cc00 0x55663c013180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.787+0000 7fc68f402f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663bd50780 con 0x55663c16cc00 2024-09-13T23:43:31.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.787+0000 7fc68d3ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16cc00 0x55663c013180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.787+0000 7fc68d3ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16cc00 0x55663c013180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52198/0 (socket says 172.21.15.167:52198) 2024-09-13T23:43:31.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.787+0000 7fc68d3ba700 1 -- 172.21.15.167:0/3667527 learned_addr learned my addr 172.21.15.167:0/3667527 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.787+0000 7fc68d3ba700 1 -- 172.21.15.167:0/3667527 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55663bd3cb60 con 0x55663c16cc00 2024-09-13T23:43:31.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68d3ba700 1 --2- 172.21.15.167:0/3667527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16cc00 0x55663c013180 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x55663c178090 tx=0x55663c08ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8dfce20572fb641 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc674388700 1 -- 172.21.15.167:0/3667527 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663c16e160 con 0x55663c16cc00 2024-09-13T23:43:31.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc674388700 1 -- 172.21.15.167:0/3667527 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55663c17a180 con 0x55663c16cc00 2024-09-13T23:43:31.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc674388700 1 -- 172.21.15.167:0/3667527 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663c16e2c0 con 0x55663c16cc00 2024-09-13T23:43:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68f402f40 1 -- 172.21.15.167:0/3667527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16cc00 msgr2=0x55663c013180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68f402f40 1 --2- 172.21.15.167:0/3667527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16cc00 0x55663c013180 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x55663c178090 tx=0x55663c08ff20 comp rx=0 tx=0).stop 2024-09-13T23:43:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68f402f40 1 -- 172.21.15.167:0/3667527 shutdown_connections 2024-09-13T23:43:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68f402f40 1 --2- 172.21.15.167:0/3667527 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16cc00 0x55663c013180 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68f402f40 1 -- 172.21.15.167:0/3667527 >> 172.21.15.167:0/3667527 conn(0x55663c16c800 msgr2=0x55663be2c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68f402f40 1 -- 172.21.15.167:0/3667527 shutdown_connections 2024-09-13T23:43:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68f402f40 1 -- 172.21.15.167:0/3667527 wait complete. 2024-09-13T23:43:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68f402f40 1 Processor -- start 2024-09-13T23:43:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68f402f40 1 -- start start 2024-09-13T23:43:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68f402f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16d000 0x55663c013700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68f402f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55663bdb8000 con 0x55663c16d000 2024-09-13T23:43:31.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68d3ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16d000 0x55663c013700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68d3ba700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16d000 0x55663c013700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52214/0 (socket says 172.21.15.167:52214) 2024-09-13T23:43:31.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68d3ba700 1 -- 172.21.15.167:0/648433669 learned_addr learned my addr 172.21.15.167:0/648433669 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68d3ba700 1 -- 172.21.15.167:0/648433669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55663bd3cd00 con 0x55663c16d000 2024-09-13T23:43:31.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.791+0000 7fc68d3ba700 1 --2- 172.21.15.167:0/648433669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16d000 0x55663c013700 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55663c178210 tx=0x55663c1520c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.795+0000 7fc672b85700 1 -- 172.21.15.167:0/648433669 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663c16e420 con 0x55663c16d000 2024-09-13T23:43:31.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.795+0000 7fc672b85700 1 -- 172.21.15.167:0/648433669 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55663c17a300 con 0x55663c16d000 2024-09-13T23:43:31.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.795+0000 7fc68f402f40 1 -- 172.21.15.167:0/648433669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55663bd3c340 con 0x55663c16d000 2024-09-13T23:43:31.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.795+0000 7fc672b85700 1 -- 172.21.15.167:0/648433669 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55663c16e160 con 0x55663c16d000 2024-09-13T23:43:31.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.795+0000 7fc68f402f40 1 -- 172.21.15.167:0/648433669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55663bd3cb60 con 0x55663c16d000 2024-09-13T23:43:31.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.795+0000 7fc672b85700 1 -- 172.21.15.167:0/648433669 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55663be25b80 con 0x55663c16d000 2024-09-13T23:43:31.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.795+0000 7fc672b85700 1 --2- 172.21.15.167:0/648433669 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c1fe000 0x55663c204000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.795+0000 7fc672b85700 1 -- 172.21.15.167:0/648433669 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55663bd916c0 con 0x55663c16d000 2024-09-13T23:43:31.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.795+0000 7fc68e3bc700 1 --2- 172.21.15.167:0/648433669 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c1fe000 0x55663c204000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.795+0000 7fc68f402f40 5 note: GC not initialized 2024-09-13T23:43:31.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.795+0000 7fc68e3bc700 1 --2- 172.21.15.167:0/648433669 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c1fe000 0x55663c204000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55663c105980 tx=0x55663c08ea80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.799+0000 7fc68f402f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:31.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.799+0000 7fc68f402f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:31.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.799+0000 7fc66e37c700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:01 2024 2024-09-13T23:43:31.807 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:31.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.799+0000 7fc66db5a700 20 reqs_thread_entry: start 2024-09-13T23:43:31.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.799+0000 7fc68f402f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.799+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c16dc00 con 0x55663c15a000 2024-09-13T23:43:31.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.799+0000 7fc670380700 20 reqs_thread_entry: start 2024-09-13T23:43:31.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.799+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv18 ondisk = 0) v8 ==== 275+0+304 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.799+0000 7fc68f402f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:31.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.799+0000 7fc674b89700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:31.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.799+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93897583108096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c1fe400 con 0x55663c0cb400 2024-09-13T23:43:31.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.803+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 93897583108096] v33'109 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.803+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93897581689600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c1fe800 con 0x55663c0cbc00 2024-09-13T23:43:31.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.807+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 93897581689600] v33'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.807+0000 7fc68f402f40 20 remove_watcher() i=0 2024-09-13T23:43:31.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.807+0000 7fc68f402f40 2 removed watcher, disabling cache 2024-09-13T23:43:31.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.807+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93897581690752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c1fec00 con 0x55663c0cb400 2024-09-13T23:43:31.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.807+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 93897581690752] v33'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c0cb400 2024-09-13T23:43:31.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.807+0000 7fc68f402f40 20 remove_watcher() i=1 2024-09-13T23:43:31.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.807+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93897581691904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c1ff000 con 0x55663c0cbc00 2024-09-13T23:43:31.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.811+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 93897581691904] v33'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.811+0000 7fc68f402f40 20 remove_watcher() i=2 2024-09-13T23:43:31.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.811+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93897581693056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c1ff400 con 0x55663c15a000 2024-09-13T23:43:31.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.815+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 93897581693056] v33'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.815+0000 7fc68f402f40 20 remove_watcher() i=3 2024-09-13T23:43:31.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.815+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93897581694208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c1ff800 con 0x55663c0cbc00 2024-09-13T23:43:31.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.815+0000 7fc68dbbb700 1 -- 172.21.15.167:0/1212737929 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 93897581694208] v33'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159200 con 0x55663c0cbc00 2024-09-13T23:43:31.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.815+0000 7fc68f402f40 20 remove_watcher() i=4 2024-09-13T23:43:31.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.815+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93897581695360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c1ffc00 con 0x55663c15a800 2024-09-13T23:43:31.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.819+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 93897581695360] v33'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.819+0000 7fc68f402f40 20 remove_watcher() i=5 2024-09-13T23:43:31.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.819+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93897581696512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c16c000 con 0x55663c15a800 2024-09-13T23:43:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 93897581696512] v33'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c159440 con 0x55663c15a800 2024-09-13T23:43:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 20 remove_watcher() i=6 2024-09-13T23:43:31.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93897581697664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55663c16c400 con 0x55663c15a000 2024-09-13T23:43:31.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68e3bc700 1 -- 172.21.15.167:0/1212737929 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 93897581697664] v33'113 uv111 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55663c158b40 con 0x55663c15a000 2024-09-13T23:43:31.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 20 remove_watcher() i=7 2024-09-13T23:43:31.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55663c0cbc00 msgr2=0x55663c012100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55663c0cbc00 0x55663c012100 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55663c0cb400 msgr2=0x55663c011b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55663c0cb400 0x55663c011b80 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55663c15a000 msgr2=0x55663c012c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55663c15a000 0x55663c012c00 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55663c15a800 msgr2=0x55663c012680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55663c15a800 0x55663c012680 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c0cb000 msgr2=0x55663c011600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c0cb000 0x55663c011600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55663caf4030 tx=0x55663c152000 comp rx=0 tx=0).stop 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 msgr2=0x55663c010b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 --2- 172.21.15.167:0/1212737929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b800 0x55663c010b00 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x55663c0c4b40 tx=0x55663c08ee40 comp rx=0 tx=0).stop 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 reap_dead start 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 shutdown_connections 2024-09-13T23:43:31.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68d3ba700 1 -- 172.21.15.167:0/1212737929 reap_dead start 2024-09-13T23:43:31.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 >> 172.21.15.167:0/1212737929 conn(0x55663bd2b400 msgr2=0x55663be2b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 shutdown_connections 2024-09-13T23:43:31.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.823+0000 7fc68f402f40 1 -- 172.21.15.167:0/1212737929 wait complete. 2024-09-13T23:43:31.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 -- 172.21.15.167:0/648433669 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c1fe000 msgr2=0x55663c204000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 --2- 172.21.15.167:0/648433669 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c1fe000 0x55663c204000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x55663c105980 tx=0x55663c08ea80 comp rx=0 tx=0).stop 2024-09-13T23:43:31.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 -- 172.21.15.167:0/648433669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16d000 msgr2=0x55663c013700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 --2- 172.21.15.167:0/648433669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16d000 0x55663c013700 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x55663c178210 tx=0x55663c1520c0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 -- 172.21.15.167:0/648433669 shutdown_connections 2024-09-13T23:43:31.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 --2- 172.21.15.167:0/648433669 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c1fe000 0x55663c204000 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 --2- 172.21.15.167:0/648433669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663c16d000 0x55663c013700 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 -- 172.21.15.167:0/648433669 >> 172.21.15.167:0/648433669 conn(0x55663c16c800 msgr2=0x55663be2c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 -- 172.21.15.167:0/648433669 shutdown_connections 2024-09-13T23:43:31.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 -- 172.21.15.167:0/648433669 wait complete. 2024-09-13T23:43:31.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 -- 172.21.15.167:0/3995724225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c0ca000 msgr2=0x55663c011080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 --2- 172.21.15.167:0/3995724225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c0ca000 0x55663c011080 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x55663c0561b0 tx=0x55663c08eb40 comp rx=0 tx=0).stop 2024-09-13T23:43:31.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 -- 172.21.15.167:0/3995724225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 msgr2=0x55663c010580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 --2- 172.21.15.167:0/3995724225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x55663caf5ef0 tx=0x55663be2f980 comp rx=0 tx=0).stop 2024-09-13T23:43:31.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 -- 172.21.15.167:0/3995724225 shutdown_connections 2024-09-13T23:43:31.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 --2- 172.21.15.167:0/3995724225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55663c0ca000 0x55663c011080 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 --2- 172.21.15.167:0/3995724225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55663bd2b000 0x55663c010580 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 -- 172.21.15.167:0/3995724225 >> 172.21.15.167:0/3995724225 conn(0x55663bd2ac00 msgr2=0x55663be2b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 -- 172.21.15.167:0/3995724225 shutdown_connections 2024-09-13T23:43:31.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.827+0000 7fc68f402f40 1 -- 172.21.15.167:0/3995724225 wait complete. 2024-09-13T23:43:31.842 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:31.842 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "bucket:myfoo", 2024-09-13T23:43:31.842 INFO:teuthology.orchestra.run.smithi167.stdout: "ver": { 2024-09-13T23:43:31.842 INFO:teuthology.orchestra.run.smithi167.stdout: "tag": "_hG32gnavFCpwqpu3cICzG6U", 2024-09-13T23:43:31.842 INFO:teuthology.orchestra.run.smithi167.stdout: "ver": 2 2024-09-13T23:43:31.842 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:31.842 INFO:teuthology.orchestra.run.smithi167.stdout: "mtime": "2024-09-13T23:43:31.402540Z", 2024-09-13T23:43:31.842 INFO:teuthology.orchestra.run.smithi167.stdout: "data": { 2024-09-13T23:43:31.842 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": { 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: "name": "myfoo", 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: "marker": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1", 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_id": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1", 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: "tenant": "", 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: "explicit_placement": { 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": "", 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "", 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": "" 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: "creation_time": "2024-09-13T23:43:29.849424Z", 2024-09-13T23:43:31.843 INFO:teuthology.orchestra.run.smithi167.stdout: "linked": false, 2024-09-13T23:43:31.844 INFO:teuthology.orchestra.run.smithi167.stdout: "has_bucket_info": false 2024-09-13T23:43:31.844 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:31.844 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:31.844 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:myfoo', 'ver': {'tag': '_hG32gnavFCpwqpu3cICzG6U', 'ver': 2}, 'mtime': '2024-09-13T23:43:31.402540Z', 'data': {'bucket': {'name': 'myfoo', 'marker': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1', 'bucket_id': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1', 'tenant': '', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'foo', 'creation_time': '2024-09-13T23:43:29.849424Z', 'linked': False, 'has_bucket_info': False}} 2024-09-13T23:43:31.844 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1'] 2024-09-13T23:43:31.844 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1'] 2024-09-13T23:43:31.844 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid fud --bucket myfoo --bucket-id 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 2024-09-13T23:43:31.890 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:31.890 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:31.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d5e68f40 1 Processor -- start 2024-09-13T23:43:31.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d5e68f40 1 -- start start 2024-09-13T23:43:31.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d5e68f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55d56a3bd400 0x55d56a6a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d5e68f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d56a481500 con 0x55d56a3bd000 2024-09-13T23:43:31.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d5e68f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d56a44aa00 con 0x55d56a3bd400 2024-09-13T23:43:31.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d3e20700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55d56a3bd400 0x55d56a6a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d4621700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55d56a3bd000 0x55d56a4bc800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45486/0 (socket says 172.21.15.167:45486) 2024-09-13T23:43:31.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d4621700 1 -- 172.21.15.167:0/3494078625 learned_addr learned my addr 172.21.15.167:0/3494078625 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:31.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d3e20700 1 -- 172.21.15.167:0/3494078625 >> v1:172.21.15.12:6789/0 conn(0x55d56a3bd000 legacy=0x55d56a4bc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d361f700 1 -- 172.21.15.167:0/3494078625 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706000383 0 0) 0x55d56a518000 con 0x55d56a3bd000 2024-09-13T23:43:31.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d3e20700 1 -- 172.21.15.167:0/3494078625 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d56a3cfd40 con 0x55d56a3bd400 2024-09-13T23:43:31.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d3e20700 1 --2- 172.21.15.167:0/3494078625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd400 0x55d56a6a2580 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55d56a69d920 tx=0x55d56a4c1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=429b32312b8ce015 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d361f700 1 -- 172.21.15.167:0/3494078625 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a4614a0 con 0x55d56a3bd400 2024-09-13T23:43:31.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d361f700 1 -- 172.21.15.167:0/3494078625 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d56a481800 con 0x55d56a3bd400 2024-09-13T23:43:31.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d361f700 1 -- 172.21.15.167:0/3494078625 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a461600 con 0x55d56a3bd400 2024-09-13T23:43:31.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3494078625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd400 msgr2=0x55d56a6a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/3494078625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd400 0x55d56a6a2580 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x55d56a69d920 tx=0x55d56a4c1920 comp rx=0 tx=0).stop 2024-09-13T23:43:31.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3494078625 shutdown_connections 2024-09-13T23:43:31.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/3494078625 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd400 0x55d56a6a2580 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3494078625 >> 172.21.15.167:0/3494078625 conn(0x55d56a3bcc00 msgr2=0x55d56a4bc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3494078625 shutdown_connections 2024-09-13T23:43:31.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.883+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3494078625 wait complete. 2024-09-13T23:43:31.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 Processor -- start 2024-09-13T23:43:31.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 -- start start 2024-09-13T23:43:31.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 0x55d56a6a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56a44a500 con 0x55d56a3bd000 2024-09-13T23:43:31.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d4621700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 0x55d56a6a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d4621700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 0x55d56a6a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52244/0 (socket says 172.21.15.167:52244) 2024-09-13T23:43:31.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d4621700 1 -- 172.21.15.167:0/1421313042 learned_addr learned my addr 172.21.15.167:0/1421313042 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d4621700 1 -- 172.21.15.167:0/1421313042 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d56a3cfba0 con 0x55d56a3bd000 2024-09-13T23:43:31.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d4621700 1 --2- 172.21.15.167:0/1421313042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 0x55d56a6a2580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55d56b187d70 tx=0x55d56a4c1b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f939246fc82daa27 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d2e1e700 1 -- 172.21.15.167:0/1421313042 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a4614a0 con 0x55d56a3bd000 2024-09-13T23:43:31.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d2e1e700 1 -- 172.21.15.167:0/1421313042 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d56a481b00 con 0x55d56a3bd000 2024-09-13T23:43:31.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d2e1e700 1 -- 172.21.15.167:0/1421313042 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a461600 con 0x55d56a3bd000 2024-09-13T23:43:31.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1421313042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 msgr2=0x55d56a6a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1421313042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 0x55d56a6a2580 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x55d56b187d70 tx=0x55d56a4c1b60 comp rx=0 tx=0).stop 2024-09-13T23:43:31.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1421313042 shutdown_connections 2024-09-13T23:43:31.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1421313042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 0x55d56a6a2580 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1421313042 >> 172.21.15.167:0/1421313042 conn(0x55d56a3bcc00 msgr2=0x55d56a4bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1421313042 shutdown_connections 2024-09-13T23:43:31.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1421313042 wait complete. 2024-09-13T23:43:31.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 Processor -- start 2024-09-13T23:43:31.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 -- start start 2024-09-13T23:43:31.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 0x55d56a6a2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.887+0000 7f73d5e68f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56a44a3c0 con 0x55d56a3bd000 2024-09-13T23:43:31.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d4621700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 0x55d56a6a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d4621700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 0x55d56a6a2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52260/0 (socket says 172.21.15.167:52260) 2024-09-13T23:43:31.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d4621700 1 -- 172.21.15.167:0/1476659197 learned_addr learned my addr 172.21.15.167:0/1476659197 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d4621700 1 -- 172.21.15.167:0/1476659197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d56a3cfa00 con 0x55d56a3bd000 2024-09-13T23:43:31.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d4621700 1 --2- 172.21.15.167:0/1476659197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 0x55d56a6a2580 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55d56b187e00 tx=0x55d56a4c1a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d161b700 1 -- 172.21.15.167:0/1476659197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a4618c0 con 0x55d56a3bd000 2024-09-13T23:43:31.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d161b700 1 -- 172.21.15.167:0/1476659197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d56a481c80 con 0x55d56a3bd000 2024-09-13T23:43:31.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d161b700 1 -- 172.21.15.167:0/1476659197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a4614a0 con 0x55d56a3bd000 2024-09-13T23:43:31.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1476659197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d56a3cf860 con 0x55d56a3bd000 2024-09-13T23:43:31.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1476659197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d56a3cf6c0 con 0x55d56a3bd000 2024-09-13T23:43:31.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d5e68f40 1 Processor -- start 2024-09-13T23:43:31.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d5e68f40 1 -- start start 2024-09-13T23:43:31.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d5e68f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 0x55d56a6a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d5e68f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56a3e2640 con 0x55d56a3bd800 2024-09-13T23:43:31.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d161b700 1 -- 172.21.15.167:0/1476659197 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55d56a4b6b00 con 0x55d56a3bd000 2024-09-13T23:43:31.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d4e22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 0x55d56a6a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d4e22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 0x55d56a6a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52264/0 (socket says 172.21.15.167:52264) 2024-09-13T23:43:31.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d4e22700 1 -- 172.21.15.167:0/1411519066 learned_addr learned my addr 172.21.15.167:0/1411519066 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d161b700 1 --2- 172.21.15.167:0/1476659197 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a766000 0x55d56a6a3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.891+0000 7f73d161b700 1 -- 172.21.15.167:0/1476659197 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55d56a423180 con 0x55d56a3bd000 2024-09-13T23:43:31.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d4621700 1 --2- 172.21.15.167:0/1476659197 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a766000 0x55d56a6a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d4e22700 1 -- 172.21.15.167:0/1411519066 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d56a3ce000 con 0x55d56a3bd800 2024-09-13T23:43:31.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d4e22700 1 --2- 172.21.15.167:0/1411519066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 0x55d56a6a2b00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55d56a73c6f0 tx=0x55d56a71cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bab5f343141c0b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73cee16700 1 -- 172.21.15.167:0/1411519066 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a4618c0 con 0x55d56a3bd800 2024-09-13T23:43:31.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73cee16700 1 -- 172.21.15.167:0/1411519066 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d56a76e000 con 0x55d56a3bd800 2024-09-13T23:43:31.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73cee16700 1 -- 172.21.15.167:0/1411519066 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a4614a0 con 0x55d56a3bd800 2024-09-13T23:43:31.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1411519066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 msgr2=0x55d56a6a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1411519066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 0x55d56a6a2b00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x55d56a73c6f0 tx=0x55d56a71cde0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d4621700 1 --2- 172.21.15.167:0/1476659197 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a766000 0x55d56a6a3080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55d56a6ed590 tx=0x55d56a71c240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1411519066 shutdown_connections 2024-09-13T23:43:31.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1411519066 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 0x55d56a6a2b00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1411519066 >> 172.21.15.167:0/1411519066 conn(0x55d56a3bd400 msgr2=0x55d56a4bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1411519066 shutdown_connections 2024-09-13T23:43:31.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1411519066 wait complete. 2024-09-13T23:43:31.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d5e68f40 1 Processor -- start 2024-09-13T23:43:31.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d5e68f40 1 -- start start 2024-09-13T23:43:31.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d5e68f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 0x55d56a6a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d5e68f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56a3e28c0 con 0x55d56a3bd800 2024-09-13T23:43:31.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d4e22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 0x55d56a6a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d4e22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 0x55d56a6a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52268/0 (socket says 172.21.15.167:52268) 2024-09-13T23:43:31.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 learned_addr learned my addr 172.21.15.167:0/1919328660 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.895+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d56a3ceea0 con 0x55d56a3bd800 2024-09-13T23:43:31.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73d4e22700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 0x55d56a6a2b00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x55d56a6ec240 tx=0x55d56a71c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73cd613700 1 -- 172.21.15.167:0/1919328660 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a461b80 con 0x55d56a3bd800 2024-09-13T23:43:31.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73cd613700 1 -- 172.21.15.167:0/1919328660 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d56a76e180 con 0x55d56a3bd800 2024-09-13T23:43:31.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d56a3ce820 con 0x55d56a3bd800 2024-09-13T23:43:31.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73cd613700 1 -- 172.21.15.167:0/1919328660 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a4618c0 con 0x55d56a3bd800 2024-09-13T23:43:31.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d56a3ce9c0 con 0x55d56a3bd800 2024-09-13T23:43:31.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73cd613700 1 -- 172.21.15.167:0/1919328660 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55d56a4b6dc0 con 0x55d56a3bd800 2024-09-13T23:43:31.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73cd613700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a766c00 0x55d56a6a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73cd613700 1 -- 172.21.15.167:0/1919328660 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55d56a423340 con 0x55d56a3bd800 2024-09-13T23:43:31.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d56a767000 0x55d56a6a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73d3e20700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a766c00 0x55d56a6a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767400 con 0x55d56a767000 2024-09-13T23:43:31.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73d4e22700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d56a767000 0x55d56a6a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.899+0000 7f73d3e20700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a766c00 0x55d56a6a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55d56a69d920 tx=0x55d56a4c1800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d4e22700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d56a767000 0x55d56a6a3b80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 20 realm 2024-09-13T23:43:31.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767000 2024-09-13T23:43:31.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:31.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767c00 con 0x55d56a767000 2024-09-13T23:43:31.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d56a767400 0x55d56a6a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767400 2024-09-13T23:43:31.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d4621700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d56a767400 0x55d56a6a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d4621700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d56a767400 0x55d56a6a4100 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.903+0000 7f73d5e68f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:31.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767c00 con 0x55d56a767400 2024-09-13T23:43:31.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d5e68f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:31.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d5e68f40 20 searching for the correct realm 2024-09-13T23:43:31.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d56a828000 0x55d56a6a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a828400 con 0x55d56a828000 2024-09-13T23:43:31.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d3e20700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d56a828000 0x55d56a6a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d3e20700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d56a828000 0x55d56a6a4680 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a828400 con 0x55d56a767400 2024-09-13T23:43:31.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d4621700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d56a828400 0x55d56a6a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828400 2024-09-13T23:43:31.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d4e22700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d56a828400 0x55d56a6a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d4e22700 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d56a828400 0x55d56a6a4c00 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.907+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767000 2024-09-13T23:43:31.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767000 2024-09-13T23:43:31.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828000 2024-09-13T23:43:31.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767000 2024-09-13T23:43:31.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767400 2024-09-13T23:43:31.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767400 2024-09-13T23:43:31.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767000 2024-09-13T23:43:31.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.911+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828400 2024-09-13T23:43:31.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.915+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.915+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767000 2024-09-13T23:43:31.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.915+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.915+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828400 2024-09-13T23:43:31.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.915+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.915+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767400 2024-09-13T23:43:31.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.915+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.915+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828000 2024-09-13T23:43:31.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.915+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.915+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767000 2024-09-13T23:43:31.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.915+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.915+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767400 2024-09-13T23:43:31.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828000 2024-09-13T23:43:31.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767000 2024-09-13T23:43:31.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767000 2024-09-13T23:43:31.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767000 2024-09-13T23:43:31.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828000 2024-09-13T23:43:31.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828400 2024-09-13T23:43:31.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.919+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767000 2024-09-13T23:43:31.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828400 2024-09-13T23:43:31.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828000 2024-09-13T23:43:31.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828400 2024-09-13T23:43:31.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767400 2024-09-13T23:43:31.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767400 2024-09-13T23:43:31.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828000 2024-09-13T23:43:31.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.923+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767400 2024-09-13T23:43:31.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a767400 2024-09-13T23:43:31.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d5e68f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:31.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d5e68f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:31.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d5e68f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:31.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d5e68f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:31.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a828800 con 0x55d56a767000 2024-09-13T23:43:31.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d5e68f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a828c00 con 0x55d56a767400 2024-09-13T23:43:31.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d5e68f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:31.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.927+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829000 con 0x55d56a767400 2024-09-13T23:43:31.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:31.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 20 zone default found 2024-09-13T23:43:31.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 4 Realm: () 2024-09-13T23:43:31.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:31.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:31.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:31.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 20 zonegroup default 2024-09-13T23:43:31.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829400 con 0x55d56a828400 2024-09-13T23:43:31.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829800 con 0x55d56a828000 2024-09-13T23:43:31.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829c00 con 0x55d56a828000 2024-09-13T23:43:31.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:31.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 20 started sync module instance, tier type = 2024-09-13T23:43:31.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:31.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.931+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829800 con 0x55d56a767400 2024-09-13T23:43:31.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.935+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.935+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94375101700864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829c00 con 0x55d56a767400 2024-09-13T23:43:31.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.935+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a828c00 con 0x55d56a767000 2024-09-13T23:43:31.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.939+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94375101700864] v33'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.939+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.939+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94375101702016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829000 con 0x55d56a767000 2024-09-13T23:43:31.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.939+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829800 con 0x55d56a767400 2024-09-13T23:43:31.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.939+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94375101702016] v33'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.939+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.939+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94375101703168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767c00 con 0x55d56a767400 2024-09-13T23:43:31.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.943+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828400 2024-09-13T23:43:31.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.943+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.943+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94375101704320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a828800 con 0x55d56a828400 2024-09-13T23:43:31.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.943+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829400 con 0x55d56a767400 2024-09-13T23:43:31.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.943+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94375101703168] v33'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.947+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94375101704320] v33'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.947+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.947+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94375101705472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a828c00 con 0x55d56a767400 2024-09-13T23:43:31.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.947+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829000 con 0x55d56a828000 2024-09-13T23:43:31.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.947+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.947+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94375101706624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a82e000 con 0x55d56a828000 2024-09-13T23:43:31.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.947+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a82e400 con 0x55d56a828000 2024-09-13T23:43:31.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.947+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94375101705472] v33'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94375101706624] v33'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94375101707776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a82e800 con 0x55d56a828000 2024-09-13T23:43:31.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a82ec00 con 0x55d56a828400 2024-09-13T23:43:31.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94375101707776] v33'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'114 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94375101708928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a82f000 con 0x55d56a828400 2024-09-13T23:43:31.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d5e68f40 20 add_watcher() i=0 2024-09-13T23:43:31.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d5e68f40 20 add_watcher() i=1 2024-09-13T23:43:31.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d5e68f40 20 add_watcher() i=2 2024-09-13T23:43:31.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d5e68f40 20 add_watcher() i=3 2024-09-13T23:43:31.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d5e68f40 20 add_watcher() i=4 2024-09-13T23:43:31.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d5e68f40 20 add_watcher() i=5 2024-09-13T23:43:31.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.951+0000 7f73d5e68f40 20 add_watcher() i=6 2024-09-13T23:43:31.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.955+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94375101708928] v33'115 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.955+0000 7f73d5e68f40 20 add_watcher() i=7 2024-09-13T23:43:31.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.955+0000 7f73d5e68f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:31.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.955+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a82f400 con 0x55d56a767000 2024-09-13T23:43:31.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.955+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.955+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94375103381504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a82f800 con 0x55d56a767000 2024-09-13T23:43:31.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94375103381504] v33'110 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:31.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:31.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 1 Processor -- start 2024-09-13T23:43:31.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73bb5ef700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:31.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 1 -- start start 2024-09-13T23:43:31.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82ec00 0x55d56a6a5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56a44a3c0 con 0x55d56a82ec00 2024-09-13T23:43:31.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d3e20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82ec00 0x55d56a6a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d3e20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82ec00 0x55d56a6a5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52282/0 (socket says 172.21.15.167:52282) 2024-09-13T23:43:31.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d3e20700 1 -- 172.21.15.167:0/4206892279 learned_addr learned my addr 172.21.15.167:0/4206892279 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d3e20700 1 -- 172.21.15.167:0/4206892279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d56a3ceb60 con 0x55d56a82ec00 2024-09-13T23:43:31.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d3e20700 1 --2- 172.21.15.167:0/4206892279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82ec00 0x55d56a6a5180 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55d56a6ed6e0 tx=0x55d56a4c1860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c629efeabef7cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73badee700 1 -- 172.21.15.167:0/4206892279 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a461760 con 0x55d56a82ec00 2024-09-13T23:43:31.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73badee700 1 -- 172.21.15.167:0/4206892279 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d56a76e600 con 0x55d56a82ec00 2024-09-13T23:43:31.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73badee700 1 -- 172.21.15.167:0/4206892279 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a848000 con 0x55d56a82ec00 2024-09-13T23:43:31.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 1 -- 172.21.15.167:0/4206892279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82ec00 msgr2=0x55d56a6a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:31.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/4206892279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82ec00 0x55d56a6a5180 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55d56a6ed6e0 tx=0x55d56a4c1860 comp rx=0 tx=0).stop 2024-09-13T23:43:31.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 1 -- 172.21.15.167:0/4206892279 shutdown_connections 2024-09-13T23:43:31.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/4206892279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82ec00 0x55d56a6a5180 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:31.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 1 -- 172.21.15.167:0/4206892279 >> 172.21.15.167:0/4206892279 conn(0x55d56a82fc00 msgr2=0x55d56a4be000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:31.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 1 -- 172.21.15.167:0/4206892279 shutdown_connections 2024-09-13T23:43:31.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.959+0000 7f73d5e68f40 1 -- 172.21.15.167:0/4206892279 wait complete. 2024-09-13T23:43:31.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73d5e68f40 1 Processor -- start 2024-09-13T23:43:31.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73d5e68f40 1 -- start start 2024-09-13T23:43:31.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73d5e68f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82f000 0x55d56a6a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73d5e68f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d56a44a500 con 0x55d56a82f000 2024-09-13T23:43:31.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73d3e20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82f000 0x55d56a6a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73d3e20700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82f000 0x55d56a6a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52294/0 (socket says 172.21.15.167:52294) 2024-09-13T23:43:31.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73d3e20700 1 -- 172.21.15.167:0/3488276584 learned_addr learned my addr 172.21.15.167:0/3488276584 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:31.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73d3e20700 1 -- 172.21.15.167:0/3488276584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d56a3ced00 con 0x55d56a82f000 2024-09-13T23:43:31.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73d3e20700 1 --2- 172.21.15.167:0/3488276584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82f000 0x55d56a6a5700 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55d56a8391d0 tx=0x55d56a840300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73b95eb700 1 -- 172.21.15.167:0/3488276584 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a848160 con 0x55d56a82f000 2024-09-13T23:43:31.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73b95eb700 1 -- 172.21.15.167:0/3488276584 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d56a76e300 con 0x55d56a82f000 2024-09-13T23:43:31.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3488276584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d56a3ce340 con 0x55d56a82f000 2024-09-13T23:43:31.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73b95eb700 1 -- 172.21.15.167:0/3488276584 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d56a8482c0 con 0x55d56a82f000 2024-09-13T23:43:31.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3488276584 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d56a3cfd40 con 0x55d56a82f000 2024-09-13T23:43:31.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.963+0000 7f73b95eb700 1 -- 172.21.15.167:0/3488276584 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55d56a4b7b80 con 0x55d56a82f000 2024-09-13T23:43:31.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73b95eb700 1 --2- 172.21.15.167:0/3488276584 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a8c0000 0x55d56a8c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:31.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73b95eb700 1 -- 172.21.15.167:0/3488276584 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55d56a423500 con 0x55d56a82f000 2024-09-13T23:43:31.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d4e22700 1 --2- 172.21.15.167:0/3488276584 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a8c0000 0x55d56a8c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:31.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d5e68f40 5 note: GC not initialized 2024-09-13T23:43:31.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d4e22700 1 --2- 172.21.15.167:0/3488276584 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a8c0000 0x55d56a8c6000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55d56a6ed7d0 tx=0x55d56a71d7a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:31.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73b6de6700 20 reqs_thread_entry: start 2024-09-13T23:43:31.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d5e68f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:31.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d5e68f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:31.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73b4de2700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:02 2024 2024-09-13T23:43:31.978 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:31.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d5e68f40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-13T23:43:31.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a82e400 con 0x55d56a828400 2024-09-13T23:43:31.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73b45c0700 20 reqs_thread_entry: start 2024-09-13T23:43:31.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d5e68f40 20 rados_obj.operate() r=0 bl.length=346 2024-09-13T23:43:31.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d5e68f40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-09-13T23:43:31.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d5e68f40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:43:31.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.967+0000 7f73d5e68f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:31.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-13T23:43:31.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a828800 con 0x55d56a828400 2024-09-13T23:43:31.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv18 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:31.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-13T23:43:31.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:31.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 : miss 2024-09-13T23:43:31.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:31.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829000 con 0x55d56a828000 2024-09-13T23:43:31.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(64 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv1 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:31.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:31.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:31.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:31.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:31.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 2024-09-13T23:43:31.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-09-13T23:43:31.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767800 con 0x55d56a828400 2024-09-13T23:43:31.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(65 foo.buckets [call] v33'19 uv17 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.971+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:66 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a8c0800 con 0x55d56a828000 2024-09-13T23:43:31.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.975+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(66 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v33'2 uv2 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.975+0000 7f73d5e68f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:31.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.975+0000 7f73d5e68f40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:31.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.975+0000 7f73d5e68f40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:31.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.975+0000 7f73d5e68f40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1, ofs0, ns] 2024-09-13T23:43:31.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.975+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94375103383808 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a8c1000 con 0x55d56a767400 2024-09-13T23:43:31.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94375101700864 notify 141733920787 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x55d56a71a820 con 0x55d56a767400 2024-09-13T23:43:31.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73cee16700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920787 cookie 94375101700864 notifier 4534 bl.length()=725 2024-09-13T23:43:31.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73cee16700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:31.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73cee16700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:31.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.0 [notify cookie 94375103383808 out=8b] v0'0 uv114 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73cee16700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:31.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73cee16700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829c00 con 0x55d56a767400 2024-09-13T23:43:31.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:31.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94375103383808 notify 141733920787 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55d56a71a9c0 con 0x55d56a767400 2024-09-13T23:43:31.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73d5e68f40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): orig_sources= new_sources= 2024-09-13T23:43:31.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73d5e68f40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): potential sources added=[] removed=[] 2024-09-13T23:43:31.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73d5e68f40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): orig_dests= new_dests= 2024-09-13T23:43:31.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73d5e68f40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): potential dests added=[] removed=[] 2024-09-13T23:43:31.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.979+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a8c1400 con 0x55d56a767000 2024-09-13T23:43:31.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(69 fud.buckets [call] v33'1 uv1 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:31.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a8c1800 con 0x55d56a828400 2024-09-13T23:43:31.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v33'20 uv20 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:31.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73d5e68f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-09-13T23:43:31.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73d5e68f40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:31.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73d5e68f40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-13T23:43:31.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94375103383808 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a8c1c00 con 0x55d56a828000 2024-09-13T23:43:31.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== watch-notify(notify (1) cookie 94375101707776 notify 141733920771 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x55d56a8889c0 con 0x55d56a828000 2024-09-13T23:43:31.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73cee16700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920771 cookie 94375101707776 notifier 4534 bl.length()=314 2024-09-13T23:43:31.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73cee16700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-09-13T23:43:31.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(71 notify.6 [notify cookie 94375103383808 out=8b] v0'0 uv114 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73cee16700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:31.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73cee16700 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a767c00 con 0x55d56a828000 2024-09-13T23:43:31.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.983+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:31.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.987+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== watch-notify(notify_complete (2) cookie 94375103383808 notify 141733920771 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55d56a888b60 con 0x55d56a828000 2024-09-13T23:43:31.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.987+0000 7f73bb5ef700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:31.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.987+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94375103381504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a8c0800 con 0x55d56a767000 2024-09-13T23:43:32.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.991+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94375103381504] v33'111 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:32.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.991+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94375101700864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a8c0400 con 0x55d56a767400 2024-09-13T23:43:32.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.991+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94375101700864] v33'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:32.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.991+0000 7f73d5e68f40 20 remove_watcher() i=0 2024-09-13T23:43:32.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.991+0000 7f73d5e68f40 2 removed watcher, disabling cache 2024-09-13T23:43:32.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.991+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94375101702016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a8c0c00 con 0x55d56a767000 2024-09-13T23:43:32.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.995+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94375101702016] v33'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a767000 2024-09-13T23:43:32.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.995+0000 7f73d5e68f40 20 remove_watcher() i=1 2024-09-13T23:43:32.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.995+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94375101703168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a8c1c00 con 0x55d56a767400 2024-09-13T23:43:32.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.999+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94375101703168] v33'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:32.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.999+0000 7f73d5e68f40 20 remove_watcher() i=2 2024-09-13T23:43:32.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.999+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94375101704320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a8c1400 con 0x55d56a828400 2024-09-13T23:43:32.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.999+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94375101704320] v33'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:32.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.999+0000 7f73d5e68f40 20 remove_watcher() i=3 2024-09-13T23:43:32.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:31.999+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94375101705472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829000 con 0x55d56a767400 2024-09-13T23:43:32.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.003+0000 7f73d4621700 1 -- 172.21.15.167:0/1919328660 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94375101705472] v33'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813200 con 0x55d56a767400 2024-09-13T23:43:32.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.003+0000 7f73d5e68f40 20 remove_watcher() i=4 2024-09-13T23:43:32.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.003+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94375101706624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a828800 con 0x55d56a828000 2024-09-13T23:43:32.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.003+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94375101706624] v33'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:32.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.003+0000 7f73d5e68f40 20 remove_watcher() i=5 2024-09-13T23:43:32.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.003+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94375101707776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829800 con 0x55d56a828000 2024-09-13T23:43:32.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.007+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94375101707776] v33'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a813440 con 0x55d56a828000 2024-09-13T23:43:32.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.007+0000 7f73d5e68f40 20 remove_watcher() i=6 2024-09-13T23:43:32.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.007+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94375101708928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d56a829400 con 0x55d56a828400 2024-09-13T23:43:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d4e22700 1 -- 172.21.15.167:0/1919328660 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94375101708928] v33'116 uv114 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d56a812b40 con 0x55d56a828400 2024-09-13T23:43:32.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 20 remove_watcher() i=7 2024-09-13T23:43:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d56a767400 msgr2=0x55d56a6a4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d56a767400 0x55d56a6a4100 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d56a767000 msgr2=0x55d56a6a3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d56a767000 0x55d56a6a3b80 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d56a828400 msgr2=0x55d56a6a4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d56a828400 0x55d56a6a4c00 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d56a828000 msgr2=0x55d56a6a4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d56a828000 0x55d56a6a4680 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a766c00 msgr2=0x55d56a6a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a766c00 0x55d56a6a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x55d56a69d920 tx=0x55d56a4c1800 comp rx=0 tx=0).stop 2024-09-13T23:43:32.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 msgr2=0x55d56a6a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d3e20700 1 -- 172.21.15.167:0/1919328660 reap_dead start 2024-09-13T23:43:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 0x55d56a6a2b00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x55d56a6ec240 tx=0x55d56a71c840 comp rx=0 tx=0).stop 2024-09-13T23:43:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 shutdown_connections 2024-09-13T23:43:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1919328660 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd800 0x55d56a6a2b00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 >> 172.21.15.167:0/1919328660 conn(0x55d56a3bd400 msgr2=0x55d56a4bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 shutdown_connections 2024-09-13T23:43:32.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1919328660 wait complete. 2024-09-13T23:43:32.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3488276584 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a8c0000 msgr2=0x55d56a8c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/3488276584 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a8c0000 0x55d56a8c6000 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x55d56a6ed7d0 tx=0x55d56a71d7a0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3488276584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82f000 msgr2=0x55d56a6a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/3488276584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82f000 0x55d56a6a5700 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x55d56a8391d0 tx=0x55d56a840300 comp rx=0 tx=0).stop 2024-09-13T23:43:32.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3488276584 shutdown_connections 2024-09-13T23:43:32.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/3488276584 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a8c0000 0x55d56a8c6000 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/3488276584 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a82f000 0x55d56a6a5700 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3488276584 >> 172.21.15.167:0/3488276584 conn(0x55d56a82fc00 msgr2=0x55d56a4be000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3488276584 shutdown_connections 2024-09-13T23:43:32.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/3488276584 wait complete. 2024-09-13T23:43:32.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1476659197 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a766000 msgr2=0x55d56a6a3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1476659197 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a766000 0x55d56a6a3080 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x55d56a6ed590 tx=0x55d56a71c240 comp rx=0 tx=0).stop 2024-09-13T23:43:32.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1476659197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 msgr2=0x55d56a6a2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1476659197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 0x55d56a6a2580 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x55d56b187e00 tx=0x55d56a4c1a40 comp rx=0 tx=0).stop 2024-09-13T23:43:32.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1476659197 shutdown_connections 2024-09-13T23:43:32.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1476659197 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d56a766000 0x55d56a6a3080 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 --2- 172.21.15.167:0/1476659197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d56a3bd000 0x55d56a6a2580 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1476659197 >> 172.21.15.167:0/1476659197 conn(0x55d56a3bcc00 msgr2=0x55d56a4bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1476659197 shutdown_connections 2024-09-13T23:43:32.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.011+0000 7f73d5e68f40 1 -- 172.21.15.167:0/1476659197 wait complete. 2024-09-13T23:43:32.027 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-09-13T23:43:32.027 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-09-13T23:43:32.028 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-09-13T23:43:32.113 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:32.113 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:32.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.103+0000 7f67a8c14f40 1 Processor -- start 2024-09-13T23:43:32.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.103+0000 7f67a8c14f40 1 -- start start 2024-09-13T23:43:32.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.103+0000 7f67a8c14f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55d304041400 0x55d304326580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.103+0000 7f67a8c14f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55d304105500 con 0x55d304041000 2024-09-13T23:43:32.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.103+0000 7f67a8c14f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55d3040cea00 con 0x55d304041400 2024-09-13T23:43:32.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.103+0000 7f67a6bcc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55d304041400 0x55d304326580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.103+0000 7f67a73cd700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55d304041000 0x55d304140800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45488/0 (socket says 172.21.15.167:45488) 2024-09-13T23:43:32.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.103+0000 7f67a73cd700 1 -- 172.21.15.167:0/2959887949 learned_addr learned my addr 172.21.15.167:0/2959887949 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:32.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a6bcc700 1 -- 172.21.15.167:0/2959887949 >> v1:172.21.15.12:6789/0 conn(0x55d304041000 legacy=0x55d304140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a6bcc700 1 -- 172.21.15.167:0/2959887949 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d304053d40 con 0x55d304041400 2024-09-13T23:43:32.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a6bcc700 1 --2- 172.21.15.167:0/2959887949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041400 0x55d304326580 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55d304321920 tx=0x55d304145920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0a803ee4ab074e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a63cb700 1 -- 172.21.15.167:0/2959887949 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3040e54a0 con 0x55d304041400 2024-09-13T23:43:32.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a63cb700 1 -- 172.21.15.167:0/2959887949 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d304105800 con 0x55d304041400 2024-09-13T23:43:32.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a63cb700 1 -- 172.21.15.167:0/2959887949 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3040e5600 con 0x55d304041400 2024-09-13T23:43:32.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2959887949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041400 msgr2=0x55d304326580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2959887949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041400 0x55d304326580 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55d304321920 tx=0x55d304145920 comp rx=0 tx=0).stop 2024-09-13T23:43:32.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2959887949 shutdown_connections 2024-09-13T23:43:32.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2959887949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041400 0x55d304326580 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2959887949 >> 172.21.15.167:0/2959887949 conn(0x55d304040c00 msgr2=0x55d304140000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2959887949 shutdown_connections 2024-09-13T23:43:32.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2959887949 wait complete. 2024-09-13T23:43:32.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a8c14f40 1 Processor -- start 2024-09-13T23:43:32.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a8c14f40 1 -- start start 2024-09-13T23:43:32.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a8c14f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041000 0x55d304326580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a8c14f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3040ce500 con 0x55d304041000 2024-09-13T23:43:32.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a73cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041000 0x55d304326580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a73cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041000 0x55d304326580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52306/0 (socket says 172.21.15.167:52306) 2024-09-13T23:43:32.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.107+0000 7f67a73cd700 1 -- 172.21.15.167:0/3837266733 learned_addr learned my addr 172.21.15.167:0/3837266733 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a73cd700 1 -- 172.21.15.167:0/3837266733 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d304053ba0 con 0x55d304041000 2024-09-13T23:43:32.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a73cd700 1 --2- 172.21.15.167:0/3837266733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041000 0x55d304326580 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x55d30419de60 tx=0x55d304145b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61605f3ef69a2fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a5bca700 1 -- 172.21.15.167:0/3837266733 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3040e51e0 con 0x55d304041000 2024-09-13T23:43:32.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a5bca700 1 -- 172.21.15.167:0/3837266733 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d304105b00 con 0x55d304041000 2024-09-13T23:43:32.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a5bca700 1 -- 172.21.15.167:0/3837266733 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3040e54a0 con 0x55d304041000 2024-09-13T23:43:32.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a8c14f40 1 -- 172.21.15.167:0/3837266733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041000 msgr2=0x55d304326580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/3837266733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041000 0x55d304326580 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x55d30419de60 tx=0x55d304145b60 comp rx=0 tx=0).stop 2024-09-13T23:43:32.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a8c14f40 1 -- 172.21.15.167:0/3837266733 shutdown_connections 2024-09-13T23:43:32.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/3837266733 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041000 0x55d304326580 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a8c14f40 1 -- 172.21.15.167:0/3837266733 >> 172.21.15.167:0/3837266733 conn(0x55d304040c00 msgr2=0x55d304141000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a8c14f40 1 -- 172.21.15.167:0/3837266733 shutdown_connections 2024-09-13T23:43:32.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a8c14f40 1 -- 172.21.15.167:0/3837266733 wait complete. 2024-09-13T23:43:32.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a8c14f40 1 Processor -- start 2024-09-13T23:43:32.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a8c14f40 1 -- start start 2024-09-13T23:43:32.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a8c14f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041400 0x55d304326b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a8c14f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3040ce3c0 con 0x55d304041400 2024-09-13T23:43:32.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a73cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041400 0x55d304326b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a73cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041400 0x55d304326b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52318/0 (socket says 172.21.15.167:52318) 2024-09-13T23:43:32.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a73cd700 1 -- 172.21.15.167:0/4199894444 learned_addr learned my addr 172.21.15.167:0/4199894444 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a73cd700 1 -- 172.21.15.167:0/4199894444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d304053a00 con 0x55d304041400 2024-09-13T23:43:32.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a73cd700 1 --2- 172.21.15.167:0/4199894444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041400 0x55d304326b00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x55d30419d710 tx=0x55d304145a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a43c7700 1 -- 172.21.15.167:0/4199894444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3040e5600 con 0x55d304041400 2024-09-13T23:43:32.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.111+0000 7f67a43c7700 1 -- 172.21.15.167:0/4199894444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d304105c80 con 0x55d304041400 2024-09-13T23:43:32.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a8c14f40 1 -- 172.21.15.167:0/4199894444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d304053860 con 0x55d304041400 2024-09-13T23:43:32.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a43c7700 1 -- 172.21.15.167:0/4199894444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3040e51e0 con 0x55d304041400 2024-09-13T23:43:32.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a8c14f40 1 -- 172.21.15.167:0/4199894444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3040536c0 con 0x55d304041400 2024-09-13T23:43:32.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a43c7700 1 -- 172.21.15.167:0/4199894444 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55d30413ab00 con 0x55d304041400 2024-09-13T23:43:32.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a8c14f40 1 Processor -- start 2024-09-13T23:43:32.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a43c7700 1 --2- 172.21.15.167:0/4199894444 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d304041c00 0x55d304327080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a43c7700 1 -- 172.21.15.167:0/4199894444 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55d3040a7180 con 0x55d304041400 2024-09-13T23:43:32.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a8c14f40 1 -- start start 2024-09-13T23:43:32.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a7bce700 1 --2- 172.21.15.167:0/4199894444 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d304041c00 0x55d304327080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a8c14f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 0x55d304327600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a8c14f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d304066640 con 0x55d304404000 2024-09-13T23:43:32.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a73cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 0x55d304327600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a73cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 0x55d304327600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52322/0 (socket says 172.21.15.167:52322) 2024-09-13T23:43:32.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a73cd700 1 -- 172.21.15.167:0/2597746777 learned_addr learned my addr 172.21.15.167:0/2597746777 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a7bce700 1 --2- 172.21.15.167:0/4199894444 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d304041c00 0x55d304327080 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55d304364900 tx=0x55d3043acde0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a73cd700 1 -- 172.21.15.167:0/2597746777 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d304052000 con 0x55d304404000 2024-09-13T23:43:32.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.115+0000 7f67a73cd700 1 --2- 172.21.15.167:0/2597746777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 0x55d304327600 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55d304364ff0 tx=0x55d3043ac240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cee468ecafbf1aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a1bc2700 1 -- 172.21.15.167:0/2597746777 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3040e54a0 con 0x55d304404000 2024-09-13T23:43:32.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a1bc2700 1 -- 172.21.15.167:0/2597746777 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d304105c80 con 0x55d304404000 2024-09-13T23:43:32.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a1bc2700 1 -- 172.21.15.167:0/2597746777 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3040e58c0 con 0x55d304404000 2024-09-13T23:43:32.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2597746777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 msgr2=0x55d304327600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2597746777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 0x55d304327600 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55d304364ff0 tx=0x55d3043ac240 comp rx=0 tx=0).stop 2024-09-13T23:43:32.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2597746777 shutdown_connections 2024-09-13T23:43:32.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2597746777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 0x55d304327600 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2597746777 >> 172.21.15.167:0/2597746777 conn(0x55d304041800 msgr2=0x55d304141800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2597746777 shutdown_connections 2024-09-13T23:43:32.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2597746777 wait complete. 2024-09-13T23:43:32.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a8c14f40 1 Processor -- start 2024-09-13T23:43:32.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a8c14f40 1 -- start start 2024-09-13T23:43:32.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a8c14f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 0x55d304327600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a8c14f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3040668c0 con 0x55d304404000 2024-09-13T23:43:32.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a73cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 0x55d304327600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a73cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 0x55d304327600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52330/0 (socket says 172.21.15.167:52330) 2024-09-13T23:43:32.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 learned_addr learned my addr 172.21.15.167:0/1798989201 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.119+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d304052ea0 con 0x55d304404000 2024-09-13T23:43:32.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a73cd700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 0x55d304327600 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55d304410c30 tx=0x55d3043acfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a03bf700 1 -- 172.21.15.167:0/1798989201 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3040e54a0 con 0x55d304404000 2024-09-13T23:43:32.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a03bf700 1 -- 172.21.15.167:0/1798989201 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d304416000 con 0x55d304404000 2024-09-13T23:43:32.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d304052820 con 0x55d304404000 2024-09-13T23:43:32.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a03bf700 1 -- 172.21.15.167:0/1798989201 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3040e58c0 con 0x55d304404000 2024-09-13T23:43:32.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3040529c0 con 0x55d304404000 2024-09-13T23:43:32.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a03bf700 1 -- 172.21.15.167:0/1798989201 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55d30413adc0 con 0x55d304404000 2024-09-13T23:43:32.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a8c14f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a03bf700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d304405800 0x55d304328680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a03bf700 1 -- 172.21.15.167:0/1798989201 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55d3040a7340 con 0x55d304404000 2024-09-13T23:43:32.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a6bcc700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d304405800 0x55d304328680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d304404400 0x55d304327b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304404800 con 0x55d304404400 2024-09-13T23:43:32.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a7bce700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d304404400 0x55d304327b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a6bcc700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d304405800 0x55d304328680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55d304321920 tx=0x55d304145800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.123+0000 7f67a7bce700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d304404400 0x55d304327b80 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 20 realm 2024-09-13T23:43:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304404c00 con 0x55d304404400 2024-09-13T23:43:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304404800 con 0x55d304404400 2024-09-13T23:43:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d304404c00 0x55d304328100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304405c00 con 0x55d304404c00 2024-09-13T23:43:32.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a73cd700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d304404c00 0x55d304328100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a73cd700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d304404c00 0x55d304328100 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:32.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.127+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482000 con 0x55d304404c00 2024-09-13T23:43:32.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a8c14f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:32.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a8c14f40 20 searching for the correct realm 2024-09-13T23:43:32.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d304482400 0x55d304328c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482400 2024-09-13T23:43:32.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a6bcc700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d304482400 0x55d304328c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a6bcc700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d304482400 0x55d304328c00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404c00 2024-09-13T23:43:32.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a73cd700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d304482000 0x55d304326580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482000 2024-09-13T23:43:32.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a7bce700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d304482000 0x55d304326580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a7bce700 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d304482000 0x55d304326580 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304482000 2024-09-13T23:43:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.131+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404400 2024-09-13T23:43:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404400 2024-09-13T23:43:32.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482400 2024-09-13T23:43:32.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404400 2024-09-13T23:43:32.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404c00 2024-09-13T23:43:32.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404c00 2024-09-13T23:43:32.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404400 2024-09-13T23:43:32.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.135+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482000 2024-09-13T23:43:32.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304482000 2024-09-13T23:43:32.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404400 2024-09-13T23:43:32.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482000 2024-09-13T23:43:32.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304482000 2024-09-13T23:43:32.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404c00 2024-09-13T23:43:32.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482400 2024-09-13T23:43:32.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404400 2024-09-13T23:43:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.139+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404c00 2024-09-13T23:43:32.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.143+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.143+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482400 2024-09-13T23:43:32.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.143+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.143+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404400 2024-09-13T23:43:32.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.143+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.143+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404400 2024-09-13T23:43:32.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.143+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.143+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404400 2024-09-13T23:43:32.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.143+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.143+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482400 2024-09-13T23:43:32.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.143+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.143+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482000 2024-09-13T23:43:32.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.147+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304482000 2024-09-13T23:43:32.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.147+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404400 2024-09-13T23:43:32.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.147+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.147+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482000 2024-09-13T23:43:32.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.147+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304482000 2024-09-13T23:43:32.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.147+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482400 2024-09-13T23:43:32.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.147+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.147+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482000 2024-09-13T23:43:32.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.147+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447ab40 con 0x55d304482000 2024-09-13T23:43:32.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.147+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404c00 2024-09-13T23:43:32.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.147+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.147+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404c00 2024-09-13T23:43:32.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304482400 2024-09-13T23:43:32.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404c00 2024-09-13T23:43:32.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404c00 2024-09-13T23:43:32.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a8c14f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:32.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a8c14f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:32.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a8c14f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:32.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a8c14f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:32.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a8c14f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482c00 con 0x55d304404400 2024-09-13T23:43:32.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a8c14f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a8c14f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.151+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483000 con 0x55d304404c00 2024-09-13T23:43:32.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:32.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483400 con 0x55d304404c00 2024-09-13T23:43:32.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:32.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 zone default found 2024-09-13T23:43:32.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 4 Realm: () 2024-09-13T23:43:32.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:32.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:32.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:32.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 zonegroup default 2024-09-13T23:43:32.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483800 con 0x55d304482000 2024-09-13T23:43:32.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d304482000 2024-09-13T23:43:32.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483c00 con 0x55d304482400 2024-09-13T23:43:32.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482c00 con 0x55d304482400 2024-09-13T23:43:32.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 started sync module instance, tier type = 2024-09-13T23:43:32.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:32.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.155+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483800 con 0x55d304404c00 2024-09-13T23:43:32.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.159+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.159+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94364796836608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304404800 con 0x55d304404c00 2024-09-13T23:43:32.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.159+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404400 2024-09-13T23:43:32.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.163+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.163+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94364796837760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483000 con 0x55d304404400 2024-09-13T23:43:32.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.163+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483400 con 0x55d304404c00 2024-09-13T23:43:32.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.163+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94364796836608] v33'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.163+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94364796837760] v33'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.163+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.163+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94364796838912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483800 con 0x55d304404c00 2024-09-13T23:43:32.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.163+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483c00 con 0x55d304482000 2024-09-13T23:43:32.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.167+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d304482000 2024-09-13T23:43:32.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.167+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94364796838912] v33'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.167+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94364796840064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482c00 con 0x55d304482000 2024-09-13T23:43:32.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.167+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304405c00 con 0x55d304404c00 2024-09-13T23:43:32.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.171+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94364796840064] v33'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d304482000 2024-09-13T23:43:32.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.171+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.171+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94364796841216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304404c00 2024-09-13T23:43:32.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.171+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483000 con 0x55d304482400 2024-09-13T23:43:32.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.171+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.171+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94364796842368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483c00 con 0x55d304482400 2024-09-13T23:43:32.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.171+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482c00 con 0x55d304482400 2024-09-13T23:43:32.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.171+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94364796841216] v33'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.175+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94364796842368] v33'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.175+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.175+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94364796843520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483000 con 0x55d304482400 2024-09-13T23:43:32.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.175+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483c00 con 0x55d304482000 2024-09-13T23:43:32.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.175+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'117 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d304482000 2024-09-13T23:43:32.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.175+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94364796844672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482c00 con 0x55d304482000 2024-09-13T23:43:32.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.175+0000 7f67a8c14f40 20 add_watcher() i=0 2024-09-13T23:43:32.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.175+0000 7f67a8c14f40 20 add_watcher() i=1 2024-09-13T23:43:32.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.175+0000 7f67a8c14f40 20 add_watcher() i=2 2024-09-13T23:43:32.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.175+0000 7f67a8c14f40 20 add_watcher() i=3 2024-09-13T23:43:32.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.175+0000 7f67a8c14f40 20 add_watcher() i=4 2024-09-13T23:43:32.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.175+0000 7f67a8c14f40 20 add_watcher() i=5 2024-09-13T23:43:32.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.179+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94364796843520] v33'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.179+0000 7f67a8c14f40 20 add_watcher() i=6 2024-09-13T23:43:32.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.179+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94364796844672] v33'118 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d304482000 2024-09-13T23:43:32.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.179+0000 7f67a8c14f40 20 add_watcher() i=7 2024-09-13T23:43:32.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.179+0000 7f67a8c14f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:32.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.179+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304492000 con 0x55d304404400 2024-09-13T23:43:32.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.179+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.179+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94364798377984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304492400 con 0x55d304404400 2024-09-13T23:43:32.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94364798377984] v33'112 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d304404400 2024-09-13T23:43:32.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f678e39b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:32.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f67a8c14f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:32.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f67a8c14f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:32.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f67a8c14f40 1 Processor -- start 2024-09-13T23:43:32.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f67a8c14f40 1 -- start start 2024-09-13T23:43:32.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f67a8c14f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 0x55d304329180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f67a8c14f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3040ce3c0 con 0x55d304492c00 2024-09-13T23:43:32.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f67a6bcc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 0x55d304329180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f67a6bcc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 0x55d304329180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52332/0 (socket says 172.21.15.167:52332) 2024-09-13T23:43:32.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f67a6bcc700 1 -- 172.21.15.167:0/3248950699 learned_addr learned my addr 172.21.15.167:0/3248950699 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f67a6bcc700 1 -- 172.21.15.167:0/3248950699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d304052b60 con 0x55d304492c00 2024-09-13T23:43:32.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f67a6bcc700 1 --2- 172.21.15.167:0/3248950699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 0x55d304329180 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55d3044113e0 tx=0x55d304145860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d51d15b37e1863 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f678db9a700 1 -- 172.21.15.167:0/3248950699 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3040e5760 con 0x55d304492c00 2024-09-13T23:43:32.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f678db9a700 1 -- 172.21.15.167:0/3248950699 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d304416900 con 0x55d304492c00 2024-09-13T23:43:32.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.183+0000 7f678db9a700 1 -- 172.21.15.167:0/3248950699 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d304494160 con 0x55d304492c00 2024-09-13T23:43:32.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 -- 172.21.15.167:0/3248950699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 msgr2=0x55d304329180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/3248950699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 0x55d304329180 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55d3044113e0 tx=0x55d304145860 comp rx=0 tx=0).stop 2024-09-13T23:43:32.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 -- 172.21.15.167:0/3248950699 shutdown_connections 2024-09-13T23:43:32.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/3248950699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 0x55d304329180 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 -- 172.21.15.167:0/3248950699 >> 172.21.15.167:0/3248950699 conn(0x55d304492800 msgr2=0x55d304142000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 -- 172.21.15.167:0/3248950699 shutdown_connections 2024-09-13T23:43:32.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 -- 172.21.15.167:0/3248950699 wait complete. 2024-09-13T23:43:32.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 Processor -- start 2024-09-13T23:43:32.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 -- start start 2024-09-13T23:43:32.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 0x55d304329180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3040ce500 con 0x55d304492c00 2024-09-13T23:43:32.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a6bcc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 0x55d304329180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a6bcc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 0x55d304329180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52334/0 (socket says 172.21.15.167:52334) 2024-09-13T23:43:32.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a6bcc700 1 -- 172.21.15.167:0/2962654667 learned_addr learned my addr 172.21.15.167:0/2962654667 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a6bcc700 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d304052d00 con 0x55d304492c00 2024-09-13T23:43:32.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a6bcc700 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 0x55d304329180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55d3044a2150 tx=0x55d3044a0300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f678c397700 1 -- 172.21.15.167:0/2962654667 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d3044942c0 con 0x55d304492c00 2024-09-13T23:43:32.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f678c397700 1 -- 172.21.15.167:0/2962654667 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d304416600 con 0x55d304492c00 2024-09-13T23:43:32.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d304052340 con 0x55d304492c00 2024-09-13T23:43:32.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f678c397700 1 -- 172.21.15.167:0/2962654667 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55d304494420 con 0x55d304492c00 2024-09-13T23:43:32.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.187+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d304053d40 con 0x55d304492c00 2024-09-13T23:43:32.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.191+0000 7f678c397700 1 -- 172.21.15.167:0/2962654667 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55d30413b8c0 con 0x55d304492c00 2024-09-13T23:43:32.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.191+0000 7f678c397700 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d30451c000 0x55d304522000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.191+0000 7f678c397700 1 -- 172.21.15.167:0/2962654667 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55d3040a76c0 con 0x55d304492c00 2024-09-13T23:43:32.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.191+0000 7f67a7bce700 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d30451c000 0x55d304522000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.191+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d304493000 0x55d304329700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.191+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304493400 con 0x55d304493000 2024-09-13T23:43:32.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.191+0000 7f67a73cd700 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d304493000 0x55d304329700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.191+0000 7f67a7bce700 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d30451c000 0x55d304522000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55d304411980 tx=0x55d3043acd20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.191+0000 7f67a73cd700 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d304493000 0x55d304329700 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.195+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'26 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.195+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d304493800 0x55d304522580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.195+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304493c00 con 0x55d304493800 2024-09-13T23:43:32.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.195+0000 7f67a6bcc700 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d304493800 0x55d304522580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.195+0000 7f67a6bcc700 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d304493800 0x55d304522580 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.195+0000 7f67a6bcc700 1 -- 172.21.15.167:0/2962654667 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'14 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55d30447b440 con 0x55d304493800 2024-09-13T23:43:32.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.195+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d30451c400 0x55d304522b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.195+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451c800 con 0x55d30451c400 2024-09-13T23:43:32.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.195+0000 7f67a7bce700 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d30451c400 0x55d304522b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.199+0000 7f67a7bce700 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d30451c400 0x55d304522b00 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.199+0000 7f67a7bce700 1 -- 172.21.15.167:0/2962654667 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'14 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d30451c400 2024-09-13T23:43:32.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.199+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d30451cc00 0x55d304523080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.199+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451d000 con 0x55d30451cc00 2024-09-13T23:43:32.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.199+0000 7f67a73cd700 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d30451cc00 0x55d304523080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.199+0000 7f67a73cd700 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d30451cc00 0x55d304523080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.203+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'113 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d30451cc00 2024-09-13T23:43:32.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.203+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451d400 con 0x55d304493000 2024-09-13T23:43:32.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.207+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'32 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.207+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451d800 con 0x55d30451c400 2024-09-13T23:43:32.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.207+0000 7f67a7bce700 1 -- 172.21.15.167:0/2962654667 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'24 uv10 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d30451c400 2024-09-13T23:43:32.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.207+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451dc00 con 0x55d304493000 2024-09-13T23:43:32.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.211+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'16 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.211+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304493400 con 0x55d30451cc00 2024-09-13T23:43:32.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.211+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'22 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d30451cc00 2024-09-13T23:43:32.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.211+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483400 con 0x55d304493000 2024-09-13T23:43:32.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.215+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.215+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483800 con 0x55d304493000 2024-09-13T23:43:32.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.219+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'21 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.219+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d304493000 2024-09-13T23:43:32.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.223+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'33 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.223+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304404800 con 0x55d30451c400 2024-09-13T23:43:32.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.223+0000 7f67a7bce700 1 -- 172.21.15.167:0/2962654667 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'16 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d30451c400 2024-09-13T23:43:32.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.223+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304405c00 con 0x55d304493000 2024-09-13T23:43:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.227+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.227+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451d000 con 0x55d30451cc00 2024-09-13T23:43:32.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.231+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'27 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d30451cc00 2024-09-13T23:43:32.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.231+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451d400 con 0x55d30451cc00 2024-09-13T23:43:32.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.231+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'114 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d30451cc00 2024-09-13T23:43:32.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.231+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451dc00 con 0x55d304493000 2024-09-13T23:43:32.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.235+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'26 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.235+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451c800 con 0x55d304493000 2024-09-13T23:43:32.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.239+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'34 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.239+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304493400 con 0x55d30451c400 2024-09-13T23:43:32.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.239+0000 7f67a7bce700 1 -- 172.21.15.167:0/2962654667 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'14 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d30451c400 2024-09-13T23:43:32.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.239+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304492000 con 0x55d304493000 2024-09-13T23:43:32.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.243+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'22 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.243+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304492400 con 0x55d30451cc00 2024-09-13T23:43:32.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.247+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'115 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d30451cc00 2024-09-13T23:43:32.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.247+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483400 con 0x55d304493000 2024-09-13T23:43:32.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.247+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'27 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.247+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483800 con 0x55d304493000 2024-09-13T23:43:32.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.251+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'24 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.251+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482800 con 0x55d30451c400 2024-09-13T23:43:32.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.255+0000 7f67a7bce700 1 -- 172.21.15.167:0/2962654667 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d30451c400 2024-09-13T23:43:32.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.255+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483c00 con 0x55d304493000 2024-09-13T23:43:32.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.259+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.259+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482c00 con 0x55d30451c400 2024-09-13T23:43:32.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.259+0000 7f67a7bce700 1 -- 172.21.15.167:0/2962654667 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'10 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d30451c400 2024-09-13T23:43:32.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.259+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483400 con 0x55d30451c400 2024-09-13T23:43:32.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.263+0000 7f67a7bce700 1 -- 172.21.15.167:0/2962654667 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'25 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d30451c400 2024-09-13T23:43:32.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.263+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304492000 con 0x55d30451c400 2024-09-13T23:43:32.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.263+0000 7f67a7bce700 1 -- 172.21.15.167:0/2962654667 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447ab40 con 0x55d30451c400 2024-09-13T23:43:32.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.267+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304492400 con 0x55d304493000 2024-09-13T23:43:32.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.267+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'14 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.267+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451d000 con 0x55d30451cc00 2024-09-13T23:43:32.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.271+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'23 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d30451cc00 2024-09-13T23:43:32.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.271+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451d400 con 0x55d304493000 2024-09-13T23:43:32.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.275+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'28 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304493000 2024-09-13T23:43:32.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.275+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451dc00 con 0x55d30451cc00 2024-09-13T23:43:32.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.275+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'28 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d30451cc00 2024-09-13T23:43:32.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.275+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451c800 con 0x55d304493800 2024-09-13T23:43:32.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.279+0000 7f67a6bcc700 1 -- 172.21.15.167:0/2962654667 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'20 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55d30447b440 con 0x55d304493800 2024-09-13T23:43:32.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.279+0000 7f6789b92700 20 reqs_thread_entry: start 2024-09-13T23:43:32.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.279+0000 7f67a8c14f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:32.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.279+0000 7f67a8c14f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:32.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.279+0000 7f6787b8e700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:02 2024 2024-09-13T23:43:32.291 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:32.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.279+0000 7f678736c700 20 reqs_thread_entry: start 2024-09-13T23:43:32.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.283+0000 7f67a8c14f40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-13T23:43:32.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.283+0000 7f67a8c14f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.283+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304482c00 con 0x55d304482000 2024-09-13T23:43:32.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.283+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55d30447ad80 con 0x55d304482000 2024-09-13T23:43:32.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.283+0000 7f67a8c14f40 20 rados_obj.operate() r=0 bl.length=346 2024-09-13T23:43:32.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.283+0000 7f67a8c14f40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-09-13T23:43:32.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.283+0000 7f67a8c14f40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:43:32.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.283+0000 7f67a8c14f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:32.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.283+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304483400 con 0x55d304404400 2024-09-13T23:43:32.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.283+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(63 fud.buckets [call out=183b] v0'0 uv1 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x55d30447ad80 con 0x55d304404400 2024-09-13T23:43:32.293 INFO:teuthology.orchestra.run.smithi167.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-09-13T23:43:32.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.283+0000 7f678e39b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:32.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.283+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94364798377984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304493400 con 0x55d304404400 2024-09-13T23:43:32.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.287+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94364798377984] v33'116 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55d30447ad80 con 0x55d304404400 2024-09-13T23:43:32.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.287+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94364796836608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304404800 con 0x55d304404c00 2024-09-13T23:43:32.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.291+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94364796836608] v33'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.291+0000 7f67a8c14f40 20 remove_watcher() i=0 2024-09-13T23:43:32.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.291+0000 7f67a8c14f40 2 removed watcher, disabling cache 2024-09-13T23:43:32.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.291+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94364796837760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d30451d800 con 0x55d304404400 2024-09-13T23:43:32.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.291+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94364796837760] v33'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447ad80 con 0x55d304404400 2024-09-13T23:43:32.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.291+0000 7f67a8c14f40 20 remove_watcher() i=1 2024-09-13T23:43:32.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.291+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94364796838912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304444000 con 0x55d304404c00 2024-09-13T23:43:32.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.295+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94364796838912] v33'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.295+0000 7f67a8c14f40 20 remove_watcher() i=2 2024-09-13T23:43:32.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.295+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94364796840064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304444400 con 0x55d304482000 2024-09-13T23:43:32.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.295+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94364796840064] v33'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447ad80 con 0x55d304482000 2024-09-13T23:43:32.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.295+0000 7f67a8c14f40 20 remove_watcher() i=3 2024-09-13T23:43:32.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.295+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94364796841216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304444800 con 0x55d304404c00 2024-09-13T23:43:32.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.299+0000 7f67a73cd700 1 -- 172.21.15.167:0/1798989201 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94364796841216] v33'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b200 con 0x55d304404c00 2024-09-13T23:43:32.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.299+0000 7f67a8c14f40 20 remove_watcher() i=4 2024-09-13T23:43:32.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.299+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94364796842368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304444c00 con 0x55d304482400 2024-09-13T23:43:32.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.303+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94364796842368] v33'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.303+0000 7f67a8c14f40 20 remove_watcher() i=5 2024-09-13T23:43:32.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.303+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94364796843520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304445000 con 0x55d304482400 2024-09-13T23:43:32.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.303+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94364796843520] v33'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447b440 con 0x55d304482400 2024-09-13T23:43:32.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.303+0000 7f67a8c14f40 20 remove_watcher() i=6 2024-09-13T23:43:32.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.303+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94364796844672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55d304445400 con 0x55d304482000 2024-09-13T23:43:32.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a7bce700 1 -- 172.21.15.167:0/1798989201 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94364796844672] v33'119 uv117 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55d30447ad80 con 0x55d304482000 2024-09-13T23:43:32.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 20 remove_watcher() i=7 2024-09-13T23:43:32.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d304404c00 msgr2=0x55d304328100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d304404c00 0x55d304328100 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d304404400 msgr2=0x55d304327b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d304404400 0x55d304327b80 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d304482000 msgr2=0x55d304326580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d304482000 0x55d304326580 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d304482400 msgr2=0x55d304328c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d304482400 0x55d304328c00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d304405800 msgr2=0x55d304328680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d304405800 0x55d304328680 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x55d304321920 tx=0x55d304145800 comp rx=0 tx=0).stop 2024-09-13T23:43:32.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 msgr2=0x55d304327600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 0x55d304327600 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55d304410c30 tx=0x55d3043acfc0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a6bcc700 1 -- 172.21.15.167:0/1798989201 reap_dead start 2024-09-13T23:43:32.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 shutdown_connections 2024-09-13T23:43:32.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/1798989201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304404000 0x55d304327600 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 >> 172.21.15.167:0/1798989201 conn(0x55d304041800 msgr2=0x55d304141800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 shutdown_connections 2024-09-13T23:43:32.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/1798989201 wait complete. 2024-09-13T23:43:32.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d304493000 msgr2=0x55d304329700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55d304493000 0x55d304329700 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d30451cc00 msgr2=0x55d304523080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55d30451cc00 0x55d304523080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d304493800 msgr2=0x55d304522580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55d304493800 0x55d304522580 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d30451c400 msgr2=0x55d304522b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55d30451c400 0x55d304522b00 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d30451c000 msgr2=0x55d304522000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d30451c000 0x55d304522000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x55d304411980 tx=0x55d3043acd20 comp rx=0 tx=0).stop 2024-09-13T23:43:32.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 msgr2=0x55d304329180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/2962654667 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304492c00 0x55d304329180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x55d3044a2150 tx=0x55d3044a0300 comp rx=0 tx=0).stop 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a73cd700 1 -- 172.21.15.167:0/2962654667 reap_dead start 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 shutdown_connections 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 >> 172.21.15.167:0/2962654667 conn(0x55d304492800 msgr2=0x55d304142000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 shutdown_connections 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.307+0000 7f67a8c14f40 1 -- 172.21.15.167:0/2962654667 wait complete. 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.311+0000 7f67a8c14f40 1 -- 172.21.15.167:0/4199894444 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d304041c00 msgr2=0x55d304327080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.311+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/4199894444 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d304041c00 0x55d304327080 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x55d304364900 tx=0x55d3043acde0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.311+0000 7f67a8c14f40 1 -- 172.21.15.167:0/4199894444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041400 msgr2=0x55d304326b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.311+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/4199894444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041400 0x55d304326b00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x55d30419d710 tx=0x55d304145a40 comp rx=0 tx=0).stop 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.311+0000 7f67a8c14f40 1 -- 172.21.15.167:0/4199894444 shutdown_connections 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.311+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/4199894444 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55d304041c00 0x55d304327080 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.311+0000 7f67a8c14f40 1 --2- 172.21.15.167:0/4199894444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55d304041400 0x55d304326b00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.311+0000 7f67a8c14f40 1 -- 172.21.15.167:0/4199894444 >> 172.21.15.167:0/4199894444 conn(0x55d304040c00 msgr2=0x55d304141000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.311+0000 7f67a8c14f40 1 -- 172.21.15.167:0/4199894444 shutdown_connections 2024-09-13T23:43:32.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.311+0000 7f67a8c14f40 1 -- 172.21.15.167:0/4199894444 wait complete. 2024-09-13T23:43:32.324 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-09-13T23:43:32.324 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-09-13T23:43:32.325 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-09-13T23:43:32.325 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid fud --bucket myfoo 2024-09-13T23:43:32.367 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:32.367 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:32.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9de877f40 1 Processor -- start 2024-09-13T23:43:32.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9de877f40 1 -- start start 2024-09-13T23:43:32.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9de877f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5601913e5400 0x5601916ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9de877f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5601914a9500 con 0x5601913e5000 2024-09-13T23:43:32.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9de877f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x560191472a00 con 0x5601913e5400 2024-09-13T23:43:32.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9dc82f700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5601913e5400 0x5601916ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9dd030700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5601913e5000 0x5601914e4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45498/0 (socket says 172.21.15.167:45498) 2024-09-13T23:43:32.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9dd030700 1 -- 172.21.15.167:0/1802039910 learned_addr learned my addr 172.21.15.167:0/1802039910 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:32.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9dc82f700 1 --2- 172.21.15.167:0/1802039910 >> v2:172.21.15.12:3300/0 conn(0x5601913e5400 0x5601916ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52342/0 (socket says 172.21.15.167:52342) 2024-09-13T23:43:32.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9dc82f700 1 -- 172.21.15.167:0/1802039910 >> v1:172.21.15.12:6789/0 conn(0x5601913e5000 legacy=0x5601914e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9dc02e700 1 -- 172.21.15.167:0/1802039910 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805473600 0 0) 0x560191540000 con 0x5601913e5000 2024-09-13T23:43:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9dc82f700 1 -- 172.21.15.167:0/1802039910 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601913f7d40 con 0x5601913e5400 2024-09-13T23:43:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9dc82f700 1 --2- 172.21.15.167:0/1802039910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5400 0x5601916ca580 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x5601916c5920 tx=0x5601914e9920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8a252c70f4e34a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9dc02e700 1 -- 172.21.15.167:0/1802039910 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5601914894a0 con 0x5601913e5400 2024-09-13T23:43:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9dc02e700 1 -- 172.21.15.167:0/1802039910 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601914a9800 con 0x5601913e5400 2024-09-13T23:43:32.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.359+0000 7fd9dc02e700 1 -- 172.21.15.167:0/1802039910 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560191489600 con 0x5601913e5400 2024-09-13T23:43:32.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9de877f40 1 -- 172.21.15.167:0/1802039910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5400 msgr2=0x5601916ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9de877f40 1 --2- 172.21.15.167:0/1802039910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5400 0x5601916ca580 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x5601916c5920 tx=0x5601914e9920 comp rx=0 tx=0).stop 2024-09-13T23:43:32.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9de877f40 1 -- 172.21.15.167:0/1802039910 shutdown_connections 2024-09-13T23:43:32.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9de877f40 1 --2- 172.21.15.167:0/1802039910 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5400 0x5601916ca580 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9de877f40 1 -- 172.21.15.167:0/1802039910 >> 172.21.15.167:0/1802039910 conn(0x5601913e4c00 msgr2=0x5601914e4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9de877f40 1 -- 172.21.15.167:0/1802039910 shutdown_connections 2024-09-13T23:43:32.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9de877f40 1 -- 172.21.15.167:0/1802039910 wait complete. 2024-09-13T23:43:32.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9de877f40 1 Processor -- start 2024-09-13T23:43:32.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9de877f40 1 -- start start 2024-09-13T23:43:32.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9de877f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5000 0x5601916ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9de877f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191472500 con 0x5601913e5000 2024-09-13T23:43:32.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9dd030700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5000 0x5601916ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9dd030700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5000 0x5601916ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52348/0 (socket says 172.21.15.167:52348) 2024-09-13T23:43:32.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9dd030700 1 -- 172.21.15.167:0/2727791172 learned_addr learned my addr 172.21.15.167:0/2727791172 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9dd030700 1 -- 172.21.15.167:0/2727791172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601913f7ba0 con 0x5601913e5000 2024-09-13T23:43:32.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9dd030700 1 --2- 172.21.15.167:0/2727791172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5000 0x5601916ca580 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x5601921afd70 tx=0x5601914e9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9daa130d9b21b1e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9db82d700 1 -- 172.21.15.167:0/2727791172 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5601914894a0 con 0x5601913e5000 2024-09-13T23:43:32.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9db82d700 1 -- 172.21.15.167:0/2727791172 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601914a9b00 con 0x5601913e5000 2024-09-13T23:43:32.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.363+0000 7fd9db82d700 1 -- 172.21.15.167:0/2727791172 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560191489600 con 0x5601913e5000 2024-09-13T23:43:32.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 -- 172.21.15.167:0/2727791172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5000 msgr2=0x5601916ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 --2- 172.21.15.167:0/2727791172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5000 0x5601916ca580 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x5601921afd70 tx=0x5601914e9b60 comp rx=0 tx=0).stop 2024-09-13T23:43:32.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 -- 172.21.15.167:0/2727791172 shutdown_connections 2024-09-13T23:43:32.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 --2- 172.21.15.167:0/2727791172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5000 0x5601916ca580 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 -- 172.21.15.167:0/2727791172 >> 172.21.15.167:0/2727791172 conn(0x5601913e4c00 msgr2=0x5601914e5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 -- 172.21.15.167:0/2727791172 shutdown_connections 2024-09-13T23:43:32.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 -- 172.21.15.167:0/2727791172 wait complete. 2024-09-13T23:43:32.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 Processor -- start 2024-09-13T23:43:32.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 -- start start 2024-09-13T23:43:32.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5400 0x5601916cab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601914723c0 con 0x5601913e5400 2024-09-13T23:43:32.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9dd030700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5400 0x5601916cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9dd030700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5400 0x5601916cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52358/0 (socket says 172.21.15.167:52358) 2024-09-13T23:43:32.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9dd030700 1 -- 172.21.15.167:0/237490321 learned_addr learned my addr 172.21.15.167:0/237490321 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9dd030700 1 -- 172.21.15.167:0/237490321 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601913f7a00 con 0x5601913e5400 2024-09-13T23:43:32.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9dd030700 1 --2- 172.21.15.167:0/237490321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5400 0x5601916cab00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x5601921afef0 tx=0x5601914e9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9da02a700 1 -- 172.21.15.167:0/237490321 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5601914898c0 con 0x5601913e5400 2024-09-13T23:43:32.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9da02a700 1 -- 172.21.15.167:0/237490321 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601914a9c80 con 0x5601913e5400 2024-09-13T23:43:32.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 -- 172.21.15.167:0/237490321 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5601913f7860 con 0x5601913e5400 2024-09-13T23:43:32.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9da02a700 1 -- 172.21.15.167:0/237490321 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5601914894a0 con 0x5601913e5400 2024-09-13T23:43:32.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.367+0000 7fd9de877f40 1 -- 172.21.15.167:0/237490321 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601913f76c0 con 0x5601913e5400 2024-09-13T23:43:32.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9de877f40 1 Processor -- start 2024-09-13T23:43:32.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9de877f40 1 -- start start 2024-09-13T23:43:32.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9de877f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 0x5601916cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9de877f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019140a640 con 0x5601913e5c00 2024-09-13T23:43:32.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9da02a700 1 -- 172.21.15.167:0/237490321 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5601914deb00 con 0x5601913e5400 2024-09-13T23:43:32.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9dd831700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 0x5601916cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9dd831700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 0x5601916cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52368/0 (socket says 172.21.15.167:52368) 2024-09-13T23:43:32.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9dd831700 1 -- 172.21.15.167:0/4211974879 learned_addr learned my addr 172.21.15.167:0/4211974879 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9da02a700 1 --2- 172.21.15.167:0/237490321 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56019179c000 0x5601916cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9dd831700 1 -- 172.21.15.167:0/4211974879 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601913f6000 con 0x5601913e5c00 2024-09-13T23:43:32.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9da02a700 1 -- 172.21.15.167:0/237490321 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56019144b180 con 0x5601913e5400 2024-09-13T23:43:32.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9dd030700 1 --2- 172.21.15.167:0/237490321 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56019179c000 0x5601916cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9dd831700 1 --2- 172.21.15.167:0/4211974879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 0x5601916cb080 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x560191711d70 tx=0x560191750960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7120a8d269d640a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9d7825700 1 -- 172.21.15.167:0/4211974879 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5601914898c0 con 0x5601913e5c00 2024-09-13T23:43:32.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9d7825700 1 -- 172.21.15.167:0/4211974879 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560191784000 con 0x5601913e5c00 2024-09-13T23:43:32.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9d7825700 1 -- 172.21.15.167:0/4211974879 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5601914894a0 con 0x5601913e5c00 2024-09-13T23:43:32.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9dd030700 1 --2- 172.21.15.167:0/237490321 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56019179c000 0x5601916cc100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5601917114d0 tx=0x560191750240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9de877f40 1 -- 172.21.15.167:0/4211974879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 msgr2=0x5601916cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9de877f40 1 --2- 172.21.15.167:0/4211974879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 0x5601916cb080 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x560191711d70 tx=0x560191750960 comp rx=0 tx=0).stop 2024-09-13T23:43:32.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9de877f40 1 -- 172.21.15.167:0/4211974879 shutdown_connections 2024-09-13T23:43:32.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9de877f40 1 --2- 172.21.15.167:0/4211974879 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 0x5601916cb080 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9de877f40 1 -- 172.21.15.167:0/4211974879 >> 172.21.15.167:0/4211974879 conn(0x5601913e5800 msgr2=0x5601914e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9de877f40 1 -- 172.21.15.167:0/4211974879 shutdown_connections 2024-09-13T23:43:32.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.371+0000 7fd9de877f40 1 -- 172.21.15.167:0/4211974879 wait complete. 2024-09-13T23:43:32.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9de877f40 1 Processor -- start 2024-09-13T23:43:32.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9de877f40 1 -- start start 2024-09-13T23:43:32.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9de877f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 0x5601916cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9de877f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56019140a8c0 con 0x5601913e5c00 2024-09-13T23:43:32.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9dd831700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 0x5601916cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9dd831700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 0x5601916cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52378/0 (socket says 172.21.15.167:52378) 2024-09-13T23:43:32.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 learned_addr learned my addr 172.21.15.167:0/2700171058 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601913f6ea0 con 0x5601913e5c00 2024-09-13T23:43:32.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9dd831700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 0x5601916cb080 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x5601916c5f80 tx=0x5601914e9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9d6022700 1 -- 172.21.15.167:0/2700171058 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560191489b80 con 0x5601913e5c00 2024-09-13T23:43:32.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9d6022700 1 -- 172.21.15.167:0/2700171058 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560191784180 con 0x5601913e5c00 2024-09-13T23:43:32.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5601913f6820 con 0x5601913e5c00 2024-09-13T23:43:32.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9d6022700 1 -- 172.21.15.167:0/2700171058 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5601914898c0 con 0x5601913e5c00 2024-09-13T23:43:32.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.375+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601913f69c0 con 0x5601913e5c00 2024-09-13T23:43:32.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9d6022700 1 -- 172.21.15.167:0/2700171058 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5601914dedc0 con 0x5601913e5c00 2024-09-13T23:43:32.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9d6022700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56019179cc00 0x5601916cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9d6022700 1 -- 172.21.15.167:0/2700171058 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56019144b340 con 0x5601913e5c00 2024-09-13T23:43:32.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9dc82f700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56019179cc00 0x5601916cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56019179d000 0x5601916cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56019179d400 con 0x56019179d000 2024-09-13T23:43:32.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9dd831700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56019179d000 0x5601916cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9dd831700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56019179d000 0x5601916cb600 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9dc82f700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56019179cc00 0x5601916cc680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5601916c5920 tx=0x5601914e9800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 20 realm 2024-09-13T23:43:32.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56019179d800 con 0x56019179d000 2024-09-13T23:43:32.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:32.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56019179dc00 con 0x56019179d000 2024-09-13T23:43:32.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56019179d400 0x5601916cbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.379+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56019179d800 con 0x56019179d400 2024-09-13T23:43:32.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9dd030700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56019179d400 0x5601916cbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9dd030700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56019179d400 0x5601916cbb80 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9de877f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:32.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x56019179dc00 con 0x56019179d400 2024-09-13T23:43:32.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9de877f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:32.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9de877f40 20 searching for the correct realm 2024-09-13T23:43:32.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9de877f40 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56019179d800 0x5601916ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56019179dc00 con 0x56019179d800 2024-09-13T23:43:32.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9dc82f700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56019179d800 0x5601916ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9dc82f700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56019179d800 0x5601916ccc00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.383+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x56019179dc00 con 0x56019179d400 2024-09-13T23:43:32.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd030700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56019179dc00 0x5601916ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179dc00 2024-09-13T23:43:32.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd831700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56019179dc00 0x5601916ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd831700 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56019179dc00 0x5601916ca580 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d000 2024-09-13T23:43:32.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d000 2024-09-13T23:43:32.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d800 2024-09-13T23:43:32.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d000 2024-09-13T23:43:32.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.387+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d400 2024-09-13T23:43:32.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.391+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.391+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d400 2024-09-13T23:43:32.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.391+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.391+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d000 2024-09-13T23:43:32.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.391+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.391+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179dc00 2024-09-13T23:43:32.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.391+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.391+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d000 2024-09-13T23:43:32.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.391+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.391+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179dc00 2024-09-13T23:43:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.391+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.391+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d400 2024-09-13T23:43:32.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d800 2024-09-13T23:43:32.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d000 2024-09-13T23:43:32.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d400 2024-09-13T23:43:32.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d800 2024-09-13T23:43:32.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d000 2024-09-13T23:43:32.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d000 2024-09-13T23:43:32.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.395+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d000 2024-09-13T23:43:32.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d800 2024-09-13T23:43:32.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179dc00 2024-09-13T23:43:32.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d000 2024-09-13T23:43:32.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179dc00 2024-09-13T23:43:32.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d800 2024-09-13T23:43:32.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179dc00 2024-09-13T23:43:32.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.399+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d400 2024-09-13T23:43:32.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d400 2024-09-13T23:43:32.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d800 2024-09-13T23:43:32.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d400 2024-09-13T23:43:32.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d400 2024-09-13T23:43:32.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9de877f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:32.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9de877f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:32.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9de877f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:32.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9de877f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:32.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.403+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852800 con 0x56019179d000 2024-09-13T23:43:32.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852c00 con 0x56019179d400 2024-09-13T23:43:32.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:32.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853000 con 0x56019179d400 2024-09-13T23:43:32.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:32.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 20 zone default found 2024-09-13T23:43:32.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 4 Realm: () 2024-09-13T23:43:32.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:32.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:32.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:32.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 20 zonegroup default 2024-09-13T23:43:32.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853400 con 0x56019179dc00 2024-09-13T23:43:32.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853800 con 0x56019179d800 2024-09-13T23:43:32.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.407+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.411+0000 7fd9de877f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.411+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.411+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853c00 con 0x56019179d800 2024-09-13T23:43:32.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.411+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.411+0000 7fd9de877f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.411+0000 7fd9de877f40 20 started sync module instance, tier type = 2024-09-13T23:43:32.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.411+0000 7fd9de877f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:32.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.411+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853800 con 0x56019179d400 2024-09-13T23:43:32.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.411+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.411+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94564734737152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853c00 con 0x56019179d400 2024-09-13T23:43:32.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.415+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852400 con 0x56019179d000 2024-09-13T23:43:32.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.415+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94564734737152] v33'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.415+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.415+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94564734738304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d000 2024-09-13T23:43:32.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.415+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852c00 con 0x56019179d400 2024-09-13T23:43:32.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.419+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94564734738304] v33'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.419+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.419+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94564734739456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853000 con 0x56019179d400 2024-09-13T23:43:32.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.419+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852800 con 0x56019179dc00 2024-09-13T23:43:32.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.423+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.423+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94564734740608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853400 con 0x56019179dc00 2024-09-13T23:43:32.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.423+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852400 con 0x56019179d400 2024-09-13T23:43:32.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.423+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94564734739456] v33'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.423+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94564734740608] v33'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.423+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.423+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94564734741760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852000 con 0x56019179d400 2024-09-13T23:43:32.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.423+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853800 con 0x56019179d800 2024-09-13T23:43:32.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.427+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.427+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94564734742912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853c00 con 0x56019179d800 2024-09-13T23:43:32.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.427+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852c00 con 0x56019179d800 2024-09-13T23:43:32.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.427+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94564734741760] v33'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.427+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94564734742912] v33'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94564734744064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853000 con 0x56019179d800 2024-09-13T23:43:32.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852400 con 0x56019179dc00 2024-09-13T23:43:32.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94564734744064] v33'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'120 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94564734745216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853800 con 0x56019179dc00 2024-09-13T23:43:32.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9de877f40 20 add_watcher() i=0 2024-09-13T23:43:32.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9de877f40 20 add_watcher() i=1 2024-09-13T23:43:32.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9de877f40 20 add_watcher() i=2 2024-09-13T23:43:32.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9de877f40 20 add_watcher() i=3 2024-09-13T23:43:32.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9de877f40 20 add_watcher() i=4 2024-09-13T23:43:32.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9de877f40 20 add_watcher() i=5 2024-09-13T23:43:32.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.431+0000 7fd9de877f40 20 add_watcher() i=6 2024-09-13T23:43:32.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94564734745216] v33'121 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9de877f40 20 add_watcher() i=7 2024-09-13T23:43:32.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9de877f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:32.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853c00 con 0x56019179d000 2024-09-13T23:43:32.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94564736434176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852c00 con 0x56019179d000 2024-09-13T23:43:32.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94564736434176] v33'117 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9c3ffe700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:32.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9de877f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:32.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9de877f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:32.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9de877f40 1 Processor -- start 2024-09-13T23:43:32.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9de877f40 1 -- start start 2024-09-13T23:43:32.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.435+0000 7fd9de877f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 0x5601916cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9de877f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601914723c0 con 0x560191853400 2024-09-13T23:43:32.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9dc82f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 0x5601916cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9dc82f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 0x5601916cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52390/0 (socket says 172.21.15.167:52390) 2024-09-13T23:43:32.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9dc82f700 1 -- 172.21.15.167:0/3614900549 learned_addr learned my addr 172.21.15.167:0/3614900549 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9dc82f700 1 -- 172.21.15.167:0/3614900549 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601913f6b60 con 0x560191853400 2024-09-13T23:43:32.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9dc82f700 1 --2- 172.21.15.167:0/3614900549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 0x5601916cd180 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x5601917f99b0 tx=0x5601914e9860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e31c877941a3c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9c37fd700 1 -- 172.21.15.167:0/3614900549 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560191489760 con 0x560191853400 2024-09-13T23:43:32.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9c37fd700 1 -- 172.21.15.167:0/3614900549 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560191784600 con 0x560191853400 2024-09-13T23:43:32.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9c37fd700 1 -- 172.21.15.167:0/3614900549 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560191874000 con 0x560191853400 2024-09-13T23:43:32.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9de877f40 1 -- 172.21.15.167:0/3614900549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 msgr2=0x5601916cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9de877f40 1 --2- 172.21.15.167:0/3614900549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 0x5601916cd180 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x5601917f99b0 tx=0x5601914e9860 comp rx=0 tx=0).stop 2024-09-13T23:43:32.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9de877f40 1 -- 172.21.15.167:0/3614900549 shutdown_connections 2024-09-13T23:43:32.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9de877f40 1 --2- 172.21.15.167:0/3614900549 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 0x5601916cd180 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9de877f40 1 -- 172.21.15.167:0/3614900549 >> 172.21.15.167:0/3614900549 conn(0x560191852800 msgr2=0x5601914e6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9de877f40 1 -- 172.21.15.167:0/3614900549 shutdown_connections 2024-09-13T23:43:32.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9de877f40 1 -- 172.21.15.167:0/3614900549 wait complete. 2024-09-13T23:43:32.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9de877f40 1 Processor -- start 2024-09-13T23:43:32.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9de877f40 1 -- start start 2024-09-13T23:43:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9de877f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 0x5601916cd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.439+0000 7fd9de877f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560191472500 con 0x560191853400 2024-09-13T23:43:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9dc82f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 0x5601916cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9dc82f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 0x5601916cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52392/0 (socket says 172.21.15.167:52392) 2024-09-13T23:43:32.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9dc82f700 1 -- 172.21.15.167:0/1591092636 learned_addr learned my addr 172.21.15.167:0/1591092636 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9dc82f700 1 -- 172.21.15.167:0/1591092636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601913f6d00 con 0x560191853400 2024-09-13T23:43:32.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9dc82f700 1 --2- 172.21.15.167:0/1591092636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 0x5601916cd180 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x56019186a1b0 tx=0x56019186c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9c1ffa700 1 -- 172.21.15.167:0/1591092636 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560191874160 con 0x560191853400 2024-09-13T23:43:32.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9c1ffa700 1 -- 172.21.15.167:0/1591092636 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560191784300 con 0x560191853400 2024-09-13T23:43:32.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9c1ffa700 1 -- 172.21.15.167:0/1591092636 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5601918742c0 con 0x560191853400 2024-09-13T23:43:32.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9de877f40 1 -- 172.21.15.167:0/1591092636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5601913f6340 con 0x560191853400 2024-09-13T23:43:32.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9de877f40 1 -- 172.21.15.167:0/1591092636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601913f7d40 con 0x560191853400 2024-09-13T23:43:32.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9c1ffa700 1 -- 172.21.15.167:0/1591092636 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5601914dfb80 con 0x560191853400 2024-09-13T23:43:32.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9c1ffa700 1 --2- 172.21.15.167:0/1591092636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5601918e6000 0x5601918ec000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9c1ffa700 1 -- 172.21.15.167:0/1591092636 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x56019144b500 con 0x560191853400 2024-09-13T23:43:32.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9de877f40 5 note: GC not initialized 2024-09-13T23:43:32.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.443+0000 7fd9dd831700 1 --2- 172.21.15.167:0/1591092636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5601918e6000 0x5601918ec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9dd831700 1 --2- 172.21.15.167:0/1591092636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5601918e6000 0x5601918ec000 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x5601917f9ce0 tx=0x5601917517a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9bf7f5700 20 reqs_thread_entry: start 2024-09-13T23:43:32.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:32.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:32.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9bd7f1700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:02 2024 2024-09-13T23:43:32.457 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:32.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9bcfcf700 20 reqs_thread_entry: start 2024-09-13T23:43:32.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-13T23:43:32.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191853c00 con 0x56019179dc00 2024-09-13T23:43:32.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 20 rados_obj.operate() r=0 bl.length=346 2024-09-13T23:43:32.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-09-13T23:43:32.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:43:32.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:32.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-13T23:43:32.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601918e6400 con 0x56019179dc00 2024-09-13T23:43:32.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv20 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:32.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-13T23:43:32.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:32.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 : miss 2024-09-13T23:43:32.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.447+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601918e6800 con 0x56019179d800 2024-09-13T23:43:32.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(64 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv2 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:32.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:32.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:32.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:32.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:32.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 2024-09-13T23:43:32.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-09-13T23:43:32.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601918e6c00 con 0x56019179d000 2024-09-13T23:43:32.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(65 fud.buckets [call] v33'2 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-09-13T23:43:32.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601918e7000 con 0x56019179dc00 2024-09-13T23:43:32.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv20 ondisk = 0) v8 ==== 233+0+288 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:32.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-09-13T23:43:32.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:32.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:32.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.451+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601918e7800 con 0x56019179dc00 2024-09-13T23:43:32.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v33'21 uv21 ondisk = 0) v8 ==== 317+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9de877f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-09-13T23:43:32.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9de877f40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:32.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9de877f40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-13T23:43:32.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94564736436480 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601918e7c00 con 0x56019179d800 2024-09-13T23:43:32.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== watch-notify(notify (1) cookie 94564734744064 notify 141733920772 ret 0) v3 ==== 380+0+0 (crc 0 0 0) 0x5601918ae9c0 con 0x56019179d800 2024-09-13T23:43:32.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9d7825700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920772 cookie 94564734744064 notifier 4550 bl.length()=338 2024-09-13T23:43:32.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(68 notify.6 [notify cookie 94564736436480 out=8b] v0'0 uv120 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9d7825700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-09-13T23:43:32.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9d7825700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:32.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9d7825700 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x560191852c00 con 0x56019179d800 2024-09-13T23:43:32.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.455+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== watch-notify(notify_complete (2) cookie 94564736436480 notify 141733920772 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5601918aeb60 con 0x56019179d800 2024-09-13T23:43:32.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.459+0000 7fd9c3ffe700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:32.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.459+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94564736434176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601918e6400 con 0x56019179d000 2024-09-13T23:43:32.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.459+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94564736434176] v33'118 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.463+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94564734737152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601918e6c00 con 0x56019179d400 2024-09-13T23:43:32.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.463+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94564734737152] v33'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.463+0000 7fd9de877f40 20 remove_watcher() i=0 2024-09-13T23:43:32.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.463+0000 7fd9de877f40 2 removed watcher, disabling cache 2024-09-13T23:43:32.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.463+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94564734738304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601918e7000 con 0x56019179d000 2024-09-13T23:43:32.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.467+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94564734738304] v33'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179d000 2024-09-13T23:43:32.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.467+0000 7fd9de877f40 20 remove_watcher() i=1 2024-09-13T23:43:32.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.467+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94564734739456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601918e7800 con 0x56019179d400 2024-09-13T23:43:32.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.467+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94564734739456] v33'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.467+0000 7fd9de877f40 20 remove_watcher() i=2 2024-09-13T23:43:32.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.467+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94564734740608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601917ba000 con 0x56019179dc00 2024-09-13T23:43:32.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.471+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94564734740608] v33'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.471+0000 7fd9de877f40 20 remove_watcher() i=3 2024-09-13T23:43:32.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.471+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94564734741760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601917ba400 con 0x56019179d400 2024-09-13T23:43:32.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.475+0000 7fd9dd030700 1 -- 172.21.15.167:0/2700171058 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94564734741760] v33'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839200 con 0x56019179d400 2024-09-13T23:43:32.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.475+0000 7fd9de877f40 20 remove_watcher() i=4 2024-09-13T23:43:32.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.475+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94564734742912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601917ba800 con 0x56019179d800 2024-09-13T23:43:32.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.475+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94564734742912] v33'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.475+0000 7fd9de877f40 20 remove_watcher() i=5 2024-09-13T23:43:32.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.475+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94564734744064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601917bac00 con 0x56019179d800 2024-09-13T23:43:32.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.479+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94564734744064] v33'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191839440 con 0x56019179d800 2024-09-13T23:43:32.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.479+0000 7fd9de877f40 20 remove_watcher() i=6 2024-09-13T23:43:32.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.479+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94564734745216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5601917bb000 con 0x56019179dc00 2024-09-13T23:43:32.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.479+0000 7fd9dd831700 1 -- 172.21.15.167:0/2700171058 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94564734745216] v33'122 uv120 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560191838b40 con 0x56019179dc00 2024-09-13T23:43:32.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.479+0000 7fd9de877f40 20 remove_watcher() i=7 2024-09-13T23:43:32.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56019179d400 msgr2=0x5601916cbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56019179d400 0x5601916cbb80 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56019179d000 msgr2=0x5601916cb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56019179d000 0x5601916cb600 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56019179dc00 msgr2=0x5601916ca580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56019179dc00 0x5601916ca580 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56019179d800 msgr2=0x5601916ccc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56019179d800 0x5601916ccc00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56019179cc00 msgr2=0x5601916cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56019179cc00 0x5601916cc680 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x5601916c5920 tx=0x5601914e9800 comp rx=0 tx=0).stop 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 msgr2=0x5601916cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 0x5601916cb080 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x5601916c5f80 tx=0x5601914e9b60 comp rx=0 tx=0).stop 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9dc82f700 1 -- 172.21.15.167:0/2700171058 reap_dead start 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 shutdown_connections 2024-09-13T23:43:32.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/2700171058 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5c00 0x5601916cb080 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 >> 172.21.15.167:0/2700171058 conn(0x5601913e5800 msgr2=0x5601914e5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 shutdown_connections 2024-09-13T23:43:32.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/2700171058 wait complete. 2024-09-13T23:43:32.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/1591092636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5601918e6000 msgr2=0x5601918ec000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/1591092636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5601918e6000 0x5601918ec000 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x5601917f9ce0 tx=0x5601917517a0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/1591092636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 msgr2=0x5601916cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/1591092636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 0x5601916cd180 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x56019186a1b0 tx=0x56019186c300 comp rx=0 tx=0).stop 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/1591092636 shutdown_connections 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/1591092636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5601918e6000 0x5601918ec000 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/1591092636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560191853400 0x5601916cd180 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/1591092636 >> 172.21.15.167:0/1591092636 conn(0x560191852800 msgr2=0x5601914e6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/1591092636 shutdown_connections 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/1591092636 wait complete. 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/237490321 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56019179c000 msgr2=0x5601916cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/237490321 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56019179c000 0x5601916cc100 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5601917114d0 tx=0x560191750240 comp rx=0 tx=0).stop 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/237490321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5400 msgr2=0x5601916cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/237490321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5400 0x5601916cab00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x5601921afef0 tx=0x5601914e9a40 comp rx=0 tx=0).stop 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/237490321 shutdown_connections 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/237490321 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56019179c000 0x5601916cc100 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 --2- 172.21.15.167:0/237490321 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5601913e5400 0x5601916cab00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/237490321 >> 172.21.15.167:0/237490321 conn(0x5601913e4c00 msgr2=0x5601914e5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/237490321 shutdown_connections 2024-09-13T23:43:32.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.483+0000 7fd9de877f40 1 -- 172.21.15.167:0/237490321 wait complete. 2024-09-13T23:43:32.499 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1'] 2024-09-13T23:43:32.499 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1'] 2024-09-13T23:43:32.499 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid foo --bucket myfoo --bucket-id 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 2024-09-13T23:43:32.545 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:32.545 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:32.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.535+0000 7fcbb7009f40 1 Processor -- start 2024-09-13T23:43:32.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.535+0000 7fcbb7009f40 1 -- start start 2024-09-13T23:43:32.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.535+0000 7fcbb7009f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ecb104b400 0x55ecb1330580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.535+0000 7fcbb7009f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ecb110f500 con 0x55ecb104b000 2024-09-13T23:43:32.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.535+0000 7fcbb7009f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ecb10d8a00 con 0x55ecb104b400 2024-09-13T23:43:32.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.535+0000 7fcbb4fc1700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ecb104b400 0x55ecb1330580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.535+0000 7fcbb57c2700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55ecb104b000 0x55ecb114a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45502/0 (socket says 172.21.15.167:45502) 2024-09-13T23:43:32.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb4fc1700 1 --2- 172.21.15.167:0/2595484481 >> v2:172.21.15.12:3300/0 conn(0x55ecb104b400 0x55ecb1330580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52406/0 (socket says 172.21.15.167:52406) 2024-09-13T23:43:32.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2595484481 learned_addr learned my addr 172.21.15.167:0/2595484481 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:32.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2595484481 >> v1:172.21.15.12:6789/0 conn(0x55ecb104b000 legacy=0x55ecb114a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2595484481 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb105dd40 con 0x55ecb104b400 2024-09-13T23:43:32.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb4fc1700 1 --2- 172.21.15.167:0/2595484481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b400 0x55ecb1330580 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55ecb132b920 tx=0x55ecb114f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc514ac748fb355 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb47c0700 1 -- 172.21.15.167:0/2595484481 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb10ef4a0 con 0x55ecb104b400 2024-09-13T23:43:32.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb47c0700 1 -- 172.21.15.167:0/2595484481 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ecb110f800 con 0x55ecb104b400 2024-09-13T23:43:32.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb47c0700 1 -- 172.21.15.167:0/2595484481 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb10ef600 con 0x55ecb104b400 2024-09-13T23:43:32.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2595484481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b400 msgr2=0x55ecb1330580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2595484481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b400 0x55ecb1330580 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x55ecb132b920 tx=0x55ecb114f920 comp rx=0 tx=0).stop 2024-09-13T23:43:32.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2595484481 shutdown_connections 2024-09-13T23:43:32.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2595484481 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b400 0x55ecb1330580 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2595484481 >> 172.21.15.167:0/2595484481 conn(0x55ecb104ac00 msgr2=0x55ecb114a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2595484481 shutdown_connections 2024-09-13T23:43:32.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2595484481 wait complete. 2024-09-13T23:43:32.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb7009f40 1 Processor -- start 2024-09-13T23:43:32.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb7009f40 1 -- start start 2024-09-13T23:43:32.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb7009f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b000 0x55ecb1330580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb7009f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb10d8500 con 0x55ecb104b000 2024-09-13T23:43:32.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb57c2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b000 0x55ecb1330580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb57c2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b000 0x55ecb1330580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52410/0 (socket says 172.21.15.167:52410) 2024-09-13T23:43:32.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.539+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2687355949 learned_addr learned my addr 172.21.15.167:0/2687355949 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2687355949 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb105dba0 con 0x55ecb104b000 2024-09-13T23:43:32.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb57c2700 1 --2- 172.21.15.167:0/2687355949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b000 0x55ecb1330580 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55ecb11a7e00 tx=0x55ecb114fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecca58847dcd28d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb3fbf700 1 -- 172.21.15.167:0/2687355949 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb10ef4a0 con 0x55ecb104b000 2024-09-13T23:43:32.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb3fbf700 1 -- 172.21.15.167:0/2687355949 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ecb110fb00 con 0x55ecb104b000 2024-09-13T23:43:32.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb3fbf700 1 -- 172.21.15.167:0/2687355949 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb10ef600 con 0x55ecb104b000 2024-09-13T23:43:32.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2687355949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b000 msgr2=0x55ecb1330580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2687355949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b000 0x55ecb1330580 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55ecb11a7e00 tx=0x55ecb114fb60 comp rx=0 tx=0).stop 2024-09-13T23:43:32.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2687355949 shutdown_connections 2024-09-13T23:43:32.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2687355949 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b000 0x55ecb1330580 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2687355949 >> 172.21.15.167:0/2687355949 conn(0x55ecb104ac00 msgr2=0x55ecb114b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2687355949 shutdown_connections 2024-09-13T23:43:32.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2687355949 wait complete. 2024-09-13T23:43:32.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb7009f40 1 Processor -- start 2024-09-13T23:43:32.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb7009f40 1 -- start start 2024-09-13T23:43:32.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb7009f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b400 0x55ecb1330b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb7009f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb10d83c0 con 0x55ecb104b400 2024-09-13T23:43:32.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb57c2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b400 0x55ecb1330b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb57c2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b400 0x55ecb1330b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52416/0 (socket says 172.21.15.167:52416) 2024-09-13T23:43:32.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb57c2700 1 -- 172.21.15.167:0/3071048330 learned_addr learned my addr 172.21.15.167:0/3071048330 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb57c2700 1 -- 172.21.15.167:0/3071048330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb105da00 con 0x55ecb104b400 2024-09-13T23:43:32.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb57c2700 1 --2- 172.21.15.167:0/3071048330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b400 0x55ecb1330b00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x55ecb11a7e00 tx=0x55ecb114fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb27bc700 1 -- 172.21.15.167:0/3071048330 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb10ef8c0 con 0x55ecb104b400 2024-09-13T23:43:32.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 -- 172.21.15.167:0/3071048330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ecb105d860 con 0x55ecb104b400 2024-09-13T23:43:32.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.543+0000 7fcbb27bc700 1 -- 172.21.15.167:0/3071048330 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ecb110fc80 con 0x55ecb104b400 2024-09-13T23:43:32.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb27bc700 1 -- 172.21.15.167:0/3071048330 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb10ef4a0 con 0x55ecb104b400 2024-09-13T23:43:32.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 -- 172.21.15.167:0/3071048330 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecb105d6c0 con 0x55ecb104b400 2024-09-13T23:43:32.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 Processor -- start 2024-09-13T23:43:32.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 -- start start 2024-09-13T23:43:32.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 0x55ecb1331080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb1070640 con 0x55ecb104bc00 2024-09-13T23:43:32.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb5fc3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 0x55ecb1331080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb5fc3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 0x55ecb1331080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52418/0 (socket says 172.21.15.167:52418) 2024-09-13T23:43:32.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2813451632 learned_addr learned my addr 172.21.15.167:0/2813451632 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb27bc700 1 -- 172.21.15.167:0/3071048330 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ecb1144b00 con 0x55ecb104b400 2024-09-13T23:43:32.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2813451632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb105c000 con 0x55ecb104bc00 2024-09-13T23:43:32.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb27bc700 1 --2- 172.21.15.167:0/3071048330 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb1404000 0x55ecb1332100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb27bc700 1 -- 172.21.15.167:0/3071048330 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ecb10b1180 con 0x55ecb104b400 2024-09-13T23:43:32.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb5fc3700 1 --2- 172.21.15.167:0/2813451632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 0x55ecb1331080 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55ecb136f110 tx=0x55ecb13ba840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c15cab9a50e71e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb57c2700 1 --2- 172.21.15.167:0/3071048330 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb1404000 0x55ecb1332100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbaffb7700 1 -- 172.21.15.167:0/2813451632 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb10ef8c0 con 0x55ecb104bc00 2024-09-13T23:43:32.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbaffb7700 1 -- 172.21.15.167:0/2813451632 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ecb13a4000 con 0x55ecb104bc00 2024-09-13T23:43:32.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbaffb7700 1 -- 172.21.15.167:0/2813451632 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb10ef4a0 con 0x55ecb104bc00 2024-09-13T23:43:32.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2813451632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 msgr2=0x55ecb1331080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2813451632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 0x55ecb1331080 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55ecb136f110 tx=0x55ecb13ba840 comp rx=0 tx=0).stop 2024-09-13T23:43:32.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb57c2700 1 --2- 172.21.15.167:0/3071048330 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb1404000 0x55ecb1332100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55ecb136f680 tx=0x55ecb13baea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2813451632 shutdown_connections 2024-09-13T23:43:32.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2813451632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 0x55ecb1331080 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2813451632 >> 172.21.15.167:0/2813451632 conn(0x55ecb104b800 msgr2=0x55ecb114b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2813451632 shutdown_connections 2024-09-13T23:43:32.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2813451632 wait complete. 2024-09-13T23:43:32.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 Processor -- start 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 -- start start 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.547+0000 7fcbb7009f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 0x55ecb1331080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbb7009f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb10708c0 con 0x55ecb104bc00 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbb5fc3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 0x55ecb1331080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbb5fc3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 0x55ecb1331080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52422/0 (socket says 172.21.15.167:52422) 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 learned_addr learned my addr 172.21.15.167:0/2837789763 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb105cea0 con 0x55ecb104bc00 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbb5fc3700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 0x55ecb1331080 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55ecb136e810 tx=0x55ecb114fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbae7b4700 1 -- 172.21.15.167:0/2837789763 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb10efb80 con 0x55ecb104bc00 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbae7b4700 1 -- 172.21.15.167:0/2837789763 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ecb13a4180 con 0x55ecb104bc00 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbae7b4700 1 -- 172.21.15.167:0/2837789763 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb10ef8c0 con 0x55ecb104bc00 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ecb105c820 con 0x55ecb104bc00 2024-09-13T23:43:32.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecb105c9c0 con 0x55ecb104bc00 2024-09-13T23:43:32.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbae7b4700 1 -- 172.21.15.167:0/2837789763 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ecb1144dc0 con 0x55ecb104bc00 2024-09-13T23:43:32.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbae7b4700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb1404c00 0x55ecb1332680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.551+0000 7fcbae7b4700 1 -- 172.21.15.167:0/2837789763 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ecb10b1340 con 0x55ecb104bc00 2024-09-13T23:43:32.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ecb1405000 0x55ecb1331600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb4fc1700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb1404c00 0x55ecb1332680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb1405400 con 0x55ecb1405000 2024-09-13T23:43:32.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb5fc3700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ecb1405000 0x55ecb1331600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb4fc1700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb1404c00 0x55ecb1332680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55ecb132b920 tx=0x55ecb114f800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb5fc3700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ecb1405000 0x55ecb1331600 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 20 realm 2024-09-13T23:43:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb1405800 con 0x55ecb1405000 2024-09-13T23:43:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb1405c00 con 0x55ecb1405000 2024-09-13T23:43:32.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ecb1405400 0x55ecb1331b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb1405800 con 0x55ecb1405400 2024-09-13T23:43:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb57c2700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ecb1405400 0x55ecb1331b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.555+0000 7fcbb57c2700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ecb1405400 0x55ecb1331b80 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb7009f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb1405c00 con 0x55ecb1405400 2024-09-13T23:43:32.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb7009f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:32.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb7009f40 20 searching for the correct realm 2024-09-13T23:43:32.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ecb1405800 0x55ecb1332c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb1405c00 con 0x55ecb1405800 2024-09-13T23:43:32.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb4fc1700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ecb1405800 0x55ecb1332c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb4fc1700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ecb1405800 0x55ecb1332c00 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb1405c00 con 0x55ecb1405400 2024-09-13T23:43:32.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb57c2700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ecb1405c00 0x55ecb1330580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405c00 2024-09-13T23:43:32.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.559+0000 7fcbb5fc3700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ecb1405c00 0x55ecb1330580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb5fc3700 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ecb1405c00 0x55ecb1330580 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405000 2024-09-13T23:43:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405000 2024-09-13T23:43:32.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405800 2024-09-13T23:43:32.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405000 2024-09-13T23:43:32.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405400 2024-09-13T23:43:32.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.563+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405400 2024-09-13T23:43:32.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405000 2024-09-13T23:43:32.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405c00 2024-09-13T23:43:32.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405000 2024-09-13T23:43:32.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405c00 2024-09-13T23:43:32.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405400 2024-09-13T23:43:32.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405800 2024-09-13T23:43:32.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.567+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405000 2024-09-13T23:43:32.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405400 2024-09-13T23:43:32.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405800 2024-09-13T23:43:32.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405000 2024-09-13T23:43:32.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405000 2024-09-13T23:43:32.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405000 2024-09-13T23:43:32.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405800 2024-09-13T23:43:32.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.571+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405c00 2024-09-13T23:43:32.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.575+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.575+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405000 2024-09-13T23:43:32.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.575+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.575+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405c00 2024-09-13T23:43:32.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.575+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.575+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405800 2024-09-13T23:43:32.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.575+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.575+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405c00 2024-09-13T23:43:32.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.575+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.575+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405400 2024-09-13T23:43:32.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.575+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.575+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405400 2024-09-13T23:43:32.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405800 2024-09-13T23:43:32.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405400 2024-09-13T23:43:32.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405400 2024-09-13T23:43:32.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb7009f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:32.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb7009f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:32.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb7009f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:32.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb7009f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:32.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba800 con 0x55ecb1405000 2024-09-13T23:43:32.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb7009f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bac00 con 0x55ecb1405400 2024-09-13T23:43:32.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb7009f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:32.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.579+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bb000 con 0x55ecb1405400 2024-09-13T23:43:32.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:32.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 20 zone default found 2024-09-13T23:43:32.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 4 Realm: () 2024-09-13T23:43:32.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:32.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:32.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:32.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 20 zonegroup default 2024-09-13T23:43:32.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bb400 con 0x55ecb1405c00 2024-09-13T23:43:32.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bb800 con 0x55ecb1405800 2024-09-13T23:43:32.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bbc00 con 0x55ecb1405800 2024-09-13T23:43:32.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 20 started sync module instance, tier type = 2024-09-13T23:43:32.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:32.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.583+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bb800 con 0x55ecb1405400 2024-09-13T23:43:32.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.587+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.587+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94475073518336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bbc00 con 0x55ecb1405400 2024-09-13T23:43:32.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.587+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba400 con 0x55ecb1405000 2024-09-13T23:43:32.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.591+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.591+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94475073519488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405000 2024-09-13T23:43:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.591+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bac00 con 0x55ecb1405400 2024-09-13T23:43:32.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.591+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94475073518336] v33'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.591+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94475073519488] v33'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.591+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.591+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94475073520640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bb000 con 0x55ecb1405400 2024-09-13T23:43:32.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.591+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba800 con 0x55ecb1405c00 2024-09-13T23:43:32.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.595+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.595+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94475073521792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bb400 con 0x55ecb1405c00 2024-09-13T23:43:32.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.595+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba400 con 0x55ecb1405400 2024-09-13T23:43:32.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.595+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94475073520640] v33'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.599+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94475073521792] v33'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.599+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.599+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94475073522944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba000 con 0x55ecb1405400 2024-09-13T23:43:32.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.599+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bb800 con 0x55ecb1405800 2024-09-13T23:43:32.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.599+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.599+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94475073524096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bbc00 con 0x55ecb1405800 2024-09-13T23:43:32.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.599+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bac00 con 0x55ecb1405800 2024-09-13T23:43:32.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.599+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94475073522944] v33'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94475073524096] v33'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94475073525248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bb000 con 0x55ecb1405800 2024-09-13T23:43:32.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14ba400 con 0x55ecb1405c00 2024-09-13T23:43:32.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94475073525248] v33'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'123 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94475073526400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bb800 con 0x55ecb1405c00 2024-09-13T23:43:32.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb7009f40 20 add_watcher() i=0 2024-09-13T23:43:32.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb7009f40 20 add_watcher() i=1 2024-09-13T23:43:32.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb7009f40 20 add_watcher() i=2 2024-09-13T23:43:32.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb7009f40 20 add_watcher() i=3 2024-09-13T23:43:32.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb7009f40 20 add_watcher() i=4 2024-09-13T23:43:32.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb7009f40 20 add_watcher() i=5 2024-09-13T23:43:32.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.603+0000 7fcbb7009f40 20 add_watcher() i=6 2024-09-13T23:43:32.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.607+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94475073526400] v33'124 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.607+0000 7fcbb7009f40 20 add_watcher() i=7 2024-09-13T23:43:32.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.607+0000 7fcbb7009f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:32.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.607+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bbc00 con 0x55ecb1405000 2024-09-13T23:43:32.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.607+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.607+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94475075215360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bac00 con 0x55ecb1405000 2024-09-13T23:43:32.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.607+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94475075215360] v33'119 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.607+0000 7fcb9c790700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:32.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:32.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:32.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 1 Processor -- start 2024-09-13T23:43:32.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 1 -- start start 2024-09-13T23:43:32.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 0x55ecb1333180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb10d83c0 con 0x55ecb14bb400 2024-09-13T23:43:32.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb4fc1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 0x55ecb1333180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb4fc1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 0x55ecb1333180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52436/0 (socket says 172.21.15.167:52436) 2024-09-13T23:43:32.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/4137269023 learned_addr learned my addr 172.21.15.167:0/4137269023 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/4137269023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb105cb60 con 0x55ecb14bb400 2024-09-13T23:43:32.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb4fc1700 1 --2- 172.21.15.167:0/4137269023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 0x55ecb1333180 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55ecb14d00f0 tx=0x55ecb114f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4f4e7ae41757e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcb9bf8f700 1 -- 172.21.15.167:0/4137269023 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb10ef760 con 0x55ecb14bb400 2024-09-13T23:43:32.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcb9bf8f700 1 -- 172.21.15.167:0/4137269023 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ecb13a4600 con 0x55ecb14bb400 2024-09-13T23:43:32.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcb9bf8f700 1 -- 172.21.15.167:0/4137269023 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb14da000 con 0x55ecb14bb400 2024-09-13T23:43:32.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 1 -- 172.21.15.167:0/4137269023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 msgr2=0x55ecb1333180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/4137269023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 0x55ecb1333180 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x55ecb14d00f0 tx=0x55ecb114f860 comp rx=0 tx=0).stop 2024-09-13T23:43:32.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 1 -- 172.21.15.167:0/4137269023 shutdown_connections 2024-09-13T23:43:32.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/4137269023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 0x55ecb1333180 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 1 -- 172.21.15.167:0/4137269023 >> 172.21.15.167:0/4137269023 conn(0x55ecb14ba800 msgr2=0x55ecb114c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 1 -- 172.21.15.167:0/4137269023 shutdown_connections 2024-09-13T23:43:32.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.611+0000 7fcbb7009f40 1 -- 172.21.15.167:0/4137269023 wait complete. 2024-09-13T23:43:32.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcbb7009f40 1 Processor -- start 2024-09-13T23:43:32.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcbb7009f40 1 -- start start 2024-09-13T23:43:32.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcbb7009f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 0x55ecb1333180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcbb7009f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ecb10d8500 con 0x55ecb14bb400 2024-09-13T23:43:32.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcbb4fc1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 0x55ecb1333180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcbb4fc1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 0x55ecb1333180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52442/0 (socket says 172.21.15.167:52442) 2024-09-13T23:43:32.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/1760478729 learned_addr learned my addr 172.21.15.167:0/1760478729 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/1760478729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ecb105cd00 con 0x55ecb14bb400 2024-09-13T23:43:32.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcbb4fc1700 1 --2- 172.21.15.167:0/1760478729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 0x55ecb1333180 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55ecb14d0390 tx=0x55ecb14d2300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcb9a78c700 1 -- 172.21.15.167:0/1760478729 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb14da160 con 0x55ecb14bb400 2024-09-13T23:43:32.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcb9a78c700 1 -- 172.21.15.167:0/1760478729 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ecb13a4300 con 0x55ecb14bb400 2024-09-13T23:43:32.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcbb7009f40 1 -- 172.21.15.167:0/1760478729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ecb105c340 con 0x55ecb14bb400 2024-09-13T23:43:32.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcb9a78c700 1 -- 172.21.15.167:0/1760478729 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ecb14da2c0 con 0x55ecb14bb400 2024-09-13T23:43:32.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcbb7009f40 1 -- 172.21.15.167:0/1760478729 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ecb105dd40 con 0x55ecb14bb400 2024-09-13T23:43:32.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcb9a78c700 1 -- 172.21.15.167:0/1760478729 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ecb1145b80 con 0x55ecb14bb400 2024-09-13T23:43:32.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcb9a78c700 1 --2- 172.21.15.167:0/1760478729 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb154a000 0x55ecb1550000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcb9a78c700 1 -- 172.21.15.167:0/1760478729 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ecb10b1500 con 0x55ecb14bb400 2024-09-13T23:43:32.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.615+0000 7fcbb7009f40 5 note: GC not initialized 2024-09-13T23:43:32.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb5fc3700 1 --2- 172.21.15.167:0/1760478729 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb154a000 0x55ecb1550000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb5fc3700 1 --2- 172.21.15.167:0/1760478729 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb154a000 0x55ecb1550000 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55ecb13dfe60 tx=0x55ecb13bb680 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcb97f87700 20 reqs_thread_entry: start 2024-09-13T23:43:32.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb7009f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:32.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb7009f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:32.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcb95f83700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:02 2024 2024-09-13T23:43:32.629 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:32.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcb95761700 20 reqs_thread_entry: start 2024-09-13T23:43:32.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb7009f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:32.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bbc00 con 0x55ecb1405000 2024-09-13T23:43:32.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb7009f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:32.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb7009f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:32.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb7009f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:32.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb7009f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:32.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb7009f40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-13T23:43:32.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.619+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb154a400 con 0x55ecb1405c00 2024-09-13T23:43:32.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv21 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:32.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-13T23:43:32.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:32.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 : miss 2024-09-13T23:43:32.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb154a800 con 0x55ecb1405800 2024-09-13T23:43:32.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(64 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv2 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:32.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:32.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:32.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:32.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:32.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 2024-09-13T23:43:32.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-09-13T23:43:32.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb154ac00 con 0x55ecb1405000 2024-09-13T23:43:32.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(65 fud.buckets [call] v33'3 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.623+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:66 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb154b400 con 0x55ecb1405800 2024-09-13T23:43:32.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(66 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v33'3 uv3 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbb7009f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:32.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbb7009f40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:32.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbb7009f40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:32.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbb7009f40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1, ofs0, ns] 2024-09-13T23:43:32.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94475075217664 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb154bc00 con 0x55ecb1405400 2024-09-13T23:43:32.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94475073518336 notify 141733920788 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x55ecb13b8820 con 0x55ecb1405400 2024-09-13T23:43:32.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbaffb7700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920788 cookie 94475073518336 notifier 4558 bl.length()=725 2024-09-13T23:43:32.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbaffb7700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:32.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbaffb7700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:32.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbaffb7700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:32.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.0 [notify cookie 94475075217664 out=8b] v0'0 uv123 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbaffb7700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb154a800 con 0x55ecb1405400 2024-09-13T23:43:32.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.627+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.631+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94475075217664 notify 141733920788 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ecb13b89c0 con 0x55ecb1405400 2024-09-13T23:43:32.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.631+0000 7fcbb7009f40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): orig_sources= new_sources= 2024-09-13T23:43:32.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.631+0000 7fcbb7009f40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): potential sources added=[] removed=[] 2024-09-13T23:43:32.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.631+0000 7fcbb7009f40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): orig_dests= new_dests= 2024-09-13T23:43:32.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.631+0000 7fcbb7009f40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): potential dests added=[] removed=[] 2024-09-13T23:43:32.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.631+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb154b000 con 0x55ecb1405c00 2024-09-13T23:43:32.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.631+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(69 foo.buckets [call] v33'22 uv22 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.631+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb154b400 con 0x55ecb1405c00 2024-09-13T23:43:32.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v33'23 uv23 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbb7009f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-09-13T23:43:32.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbb7009f40 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:32.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbb7009f40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-13T23:43:32.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94475075217664 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb154a400 con 0x55ecb1405800 2024-09-13T23:43:32.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== watch-notify(notify (1) cookie 94475073525248 notify 141733920773 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x55ecb15169c0 con 0x55ecb1405800 2024-09-13T23:43:32.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbaf7b6700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920773 cookie 94475073525248 notifier 4558 bl.length()=314 2024-09-13T23:43:32.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbaf7b6700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-09-13T23:43:32.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbaf7b6700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:32.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(71 notify.6 [notify cookie 94475075217664 out=8b] v0'0 uv123 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbaf7b6700 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14b8000 con 0x55ecb1405800 2024-09-13T23:43:32.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== watch-notify(notify_complete (2) cookie 94475075217664 notify 141733920773 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ecb1516b60 con 0x55ecb1405800 2024-09-13T23:43:32.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.635+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.639+0000 7fcb9c790700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:32.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.639+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94475075215360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb154ac00 con 0x55ecb1405000 2024-09-13T23:43:32.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.639+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94475075215360] v33'120 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.639+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94475073518336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb154b000 con 0x55ecb1405400 2024-09-13T23:43:32.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.643+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94475073518336] v33'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.643+0000 7fcbb7009f40 20 remove_watcher() i=0 2024-09-13T23:43:32.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.643+0000 7fcbb7009f40 2 removed watcher, disabling cache 2024-09-13T23:43:32.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.643+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94475073519488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb154b400 con 0x55ecb1405000 2024-09-13T23:43:32.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.643+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94475073519488] v33'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405000 2024-09-13T23:43:32.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.647+0000 7fcbb7009f40 20 remove_watcher() i=1 2024-09-13T23:43:32.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.647+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94475073520640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bac00 con 0x55ecb1405400 2024-09-13T23:43:32.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.647+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94475073520640] v33'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.647+0000 7fcbb7009f40 20 remove_watcher() i=2 2024-09-13T23:43:32.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.647+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94475073521792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14bbc00 con 0x55ecb1405c00 2024-09-13T23:43:32.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.651+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94475073521792] v33'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.651+0000 7fcbb7009f40 20 remove_watcher() i=3 2024-09-13T23:43:32.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.651+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94475073522944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14b8400 con 0x55ecb1405400 2024-09-13T23:43:32.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.651+0000 7fcbb57c2700 1 -- 172.21.15.167:0/2837789763 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94475073522944] v33'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5200 con 0x55ecb1405400 2024-09-13T23:43:32.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.651+0000 7fcbb7009f40 20 remove_watcher() i=4 2024-09-13T23:43:32.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.651+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94475073524096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14b8800 con 0x55ecb1405800 2024-09-13T23:43:32.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.655+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94475073524096] v33'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.655+0000 7fcbb7009f40 20 remove_watcher() i=5 2024-09-13T23:43:32.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.655+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94475073525248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14b8c00 con 0x55ecb1405800 2024-09-13T23:43:32.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.655+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94475073525248] v33'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a5440 con 0x55ecb1405800 2024-09-13T23:43:32.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.655+0000 7fcbb7009f40 20 remove_watcher() i=6 2024-09-13T23:43:32.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.655+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94475073526400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ecb14b9000 con 0x55ecb1405c00 2024-09-13T23:43:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb5fc3700 1 -- 172.21.15.167:0/2837789763 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94475073526400] v33'125 uv123 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ecb14a4b40 con 0x55ecb1405c00 2024-09-13T23:43:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 20 remove_watcher() i=7 2024-09-13T23:43:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ecb1405400 msgr2=0x55ecb1331b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ecb1405400 0x55ecb1331b80 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ecb1405000 msgr2=0x55ecb1331600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ecb1405000 0x55ecb1331600 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ecb1405c00 msgr2=0x55ecb1330580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ecb1405c00 0x55ecb1330580 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ecb1405800 msgr2=0x55ecb1332c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ecb1405800 0x55ecb1332c00 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb1404c00 msgr2=0x55ecb1332680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb1404c00 0x55ecb1332680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x55ecb132b920 tx=0x55ecb114f800 comp rx=0 tx=0).stop 2024-09-13T23:43:32.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 msgr2=0x55ecb1331080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 0x55ecb1331080 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55ecb136e810 tx=0x55ecb114fda0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb4fc1700 1 -- 172.21.15.167:0/2837789763 reap_dead start 2024-09-13T23:43:32.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 shutdown_connections 2024-09-13T23:43:32.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/2837789763 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104bc00 0x55ecb1331080 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 >> 172.21.15.167:0/2837789763 conn(0x55ecb104b800 msgr2=0x55ecb114b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 shutdown_connections 2024-09-13T23:43:32.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 -- 172.21.15.167:0/2837789763 wait complete. 2024-09-13T23:43:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 -- 172.21.15.167:0/1760478729 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb154a000 msgr2=0x55ecb1550000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/1760478729 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb154a000 0x55ecb1550000 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55ecb13dfe60 tx=0x55ecb13bb680 comp rx=0 tx=0).stop 2024-09-13T23:43:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 -- 172.21.15.167:0/1760478729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 msgr2=0x55ecb1333180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.659+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/1760478729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 0x55ecb1333180 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55ecb14d0390 tx=0x55ecb14d2300 comp rx=0 tx=0).stop 2024-09-13T23:43:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 -- 172.21.15.167:0/1760478729 shutdown_connections 2024-09-13T23:43:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/1760478729 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb154a000 0x55ecb1550000 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/1760478729 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb14bb400 0x55ecb1333180 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 -- 172.21.15.167:0/1760478729 >> 172.21.15.167:0/1760478729 conn(0x55ecb14ba800 msgr2=0x55ecb114c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 -- 172.21.15.167:0/1760478729 shutdown_connections 2024-09-13T23:43:32.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 -- 172.21.15.167:0/1760478729 wait complete. 2024-09-13T23:43:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 -- 172.21.15.167:0/3071048330 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb1404000 msgr2=0x55ecb1332100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/3071048330 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb1404000 0x55ecb1332100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x55ecb136f680 tx=0x55ecb13baea0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 -- 172.21.15.167:0/3071048330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b400 msgr2=0x55ecb1330b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/3071048330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b400 0x55ecb1330b00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x55ecb11a7e00 tx=0x55ecb114fa40 comp rx=0 tx=0).stop 2024-09-13T23:43:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 -- 172.21.15.167:0/3071048330 shutdown_connections 2024-09-13T23:43:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/3071048330 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ecb1404000 0x55ecb1332100 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 --2- 172.21.15.167:0/3071048330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ecb104b400 0x55ecb1330b00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 -- 172.21.15.167:0/3071048330 >> 172.21.15.167:0/3071048330 conn(0x55ecb104ac00 msgr2=0x55ecb114b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 -- 172.21.15.167:0/3071048330 shutdown_connections 2024-09-13T23:43:32.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.663+0000 7fcbb7009f40 1 -- 172.21.15.167:0/3071048330 wait complete. 2024-09-13T23:43:32.677 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-09-13T23:43:32.677 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-09-13T23:43:32.677 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-09-13T23:43:32.717 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:32.717 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:32.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.707+0000 7f4650efcf40 1 Processor -- start 2024-09-13T23:43:32.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.707+0000 7f4650efcf40 1 -- start start 2024-09-13T23:43:32.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.707+0000 7f4650efcf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x559f738b3400 0x559f73b98580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.707+0000 7f4650efcf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x559f73977500 con 0x559f738b3000 2024-09-13T23:43:32.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.707+0000 7f4650efcf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x559f73940a00 con 0x559f738b3400 2024-09-13T23:43:32.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.707+0000 7f464eeb4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x559f738b3400 0x559f73b98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.707+0000 7f464f6b5700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x559f738b3000 0x559f739b2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45510/0 (socket says 172.21.15.167:45510) 2024-09-13T23:43:32.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.707+0000 7f464eeb4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x559f738b3400 0x559f73b98580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52452/0 (socket says 172.21.15.167:52452) 2024-09-13T23:43:32.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.707+0000 7f464f6b5700 1 -- 172.21.15.167:0/925512330 learned_addr learned my addr 172.21.15.167:0/925512330 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:32.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f464eeb4700 1 -- 172.21.15.167:0/925512330 >> v1:172.21.15.12:6789/0 conn(0x559f738b3000 legacy=0x559f739b2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f464eeb4700 1 -- 172.21.15.167:0/925512330 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f738c5d40 con 0x559f738b3400 2024-09-13T23:43:32.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f464eeb4700 1 --2- 172.21.15.167:0/925512330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3400 0x559f73b98580 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x559f73b93920 tx=0x559f739b7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff0625c7dd2891f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f464e6b3700 1 -- 172.21.15.167:0/925512330 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f739574a0 con 0x559f738b3400 2024-09-13T23:43:32.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f464e6b3700 1 -- 172.21.15.167:0/925512330 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559f73977800 con 0x559f738b3400 2024-09-13T23:43:32.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f464e6b3700 1 -- 172.21.15.167:0/925512330 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f73957600 con 0x559f738b3400 2024-09-13T23:43:32.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f4650efcf40 1 -- 172.21.15.167:0/925512330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3400 msgr2=0x559f73b98580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f4650efcf40 1 --2- 172.21.15.167:0/925512330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3400 0x559f73b98580 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x559f73b93920 tx=0x559f739b7920 comp rx=0 tx=0).stop 2024-09-13T23:43:32.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f4650efcf40 1 -- 172.21.15.167:0/925512330 shutdown_connections 2024-09-13T23:43:32.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f4650efcf40 1 --2- 172.21.15.167:0/925512330 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3400 0x559f73b98580 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f4650efcf40 1 -- 172.21.15.167:0/925512330 >> 172.21.15.167:0/925512330 conn(0x559f738b2c00 msgr2=0x559f739b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f4650efcf40 1 -- 172.21.15.167:0/925512330 shutdown_connections 2024-09-13T23:43:32.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f4650efcf40 1 -- 172.21.15.167:0/925512330 wait complete. 2024-09-13T23:43:32.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f4650efcf40 1 Processor -- start 2024-09-13T23:43:32.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f4650efcf40 1 -- start start 2024-09-13T23:43:32.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f4650efcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 0x559f73b98580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f4650efcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f73940500 con 0x559f738b3000 2024-09-13T23:43:32.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f464f6b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 0x559f73b98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f464f6b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 0x559f73b98580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52456/0 (socket says 172.21.15.167:52456) 2024-09-13T23:43:32.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.711+0000 7f464f6b5700 1 -- 172.21.15.167:0/3852403715 learned_addr learned my addr 172.21.15.167:0/3852403715 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f464f6b5700 1 -- 172.21.15.167:0/3852403715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f738c5ba0 con 0x559f738b3000 2024-09-13T23:43:32.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f464f6b5700 1 --2- 172.21.15.167:0/3852403715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 0x559f73b98580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x559f73a0fe90 tx=0x559f739b7b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84465df73e6ee19f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f464deb2700 1 -- 172.21.15.167:0/3852403715 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f739571e0 con 0x559f738b3000 2024-09-13T23:43:32.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f464deb2700 1 -- 172.21.15.167:0/3852403715 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559f73977b00 con 0x559f738b3000 2024-09-13T23:43:32.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f464deb2700 1 -- 172.21.15.167:0/3852403715 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f739574a0 con 0x559f738b3000 2024-09-13T23:43:32.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f4650efcf40 1 -- 172.21.15.167:0/3852403715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 msgr2=0x559f73b98580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f4650efcf40 1 --2- 172.21.15.167:0/3852403715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 0x559f73b98580 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x559f73a0fe90 tx=0x559f739b7b60 comp rx=0 tx=0).stop 2024-09-13T23:43:32.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f4650efcf40 1 -- 172.21.15.167:0/3852403715 shutdown_connections 2024-09-13T23:43:32.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f4650efcf40 1 --2- 172.21.15.167:0/3852403715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 0x559f73b98580 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f4650efcf40 1 -- 172.21.15.167:0/3852403715 >> 172.21.15.167:0/3852403715 conn(0x559f738b2c00 msgr2=0x559f739b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f4650efcf40 1 -- 172.21.15.167:0/3852403715 shutdown_connections 2024-09-13T23:43:32.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f4650efcf40 1 -- 172.21.15.167:0/3852403715 wait complete. 2024-09-13T23:43:32.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f4650efcf40 1 Processor -- start 2024-09-13T23:43:32.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f4650efcf40 1 -- start start 2024-09-13T23:43:32.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f4650efcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 0x559f73b98580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f4650efcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f739403c0 con 0x559f738b3000 2024-09-13T23:43:32.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f464f6b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 0x559f73b98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f464f6b5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 0x559f73b98580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52462/0 (socket says 172.21.15.167:52462) 2024-09-13T23:43:32.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f464f6b5700 1 -- 172.21.15.167:0/1086323381 learned_addr learned my addr 172.21.15.167:0/1086323381 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f464f6b5700 1 -- 172.21.15.167:0/1086323381 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f738c5a00 con 0x559f738b3000 2024-09-13T23:43:32.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.715+0000 7f464f6b5700 1 --2- 172.21.15.167:0/1086323381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 0x559f73b98580 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x559f747240c0 tx=0x559f739b7a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464c6af700 1 -- 172.21.15.167:0/1086323381 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f73957600 con 0x559f738b3000 2024-09-13T23:43:32.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464c6af700 1 -- 172.21.15.167:0/1086323381 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559f73977c80 con 0x559f738b3000 2024-09-13T23:43:32.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f4650efcf40 1 -- 172.21.15.167:0/1086323381 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559f738c5860 con 0x559f738b3000 2024-09-13T23:43:32.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464c6af700 1 -- 172.21.15.167:0/1086323381 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f739571e0 con 0x559f738b3000 2024-09-13T23:43:32.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f4650efcf40 1 -- 172.21.15.167:0/1086323381 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f738c56c0 con 0x559f738b3000 2024-09-13T23:43:32.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f4650efcf40 1 Processor -- start 2024-09-13T23:43:32.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f4650efcf40 1 -- start start 2024-09-13T23:43:32.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f4650efcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 0x559f73b98b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f4650efcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f738d8640 con 0x559f738b3800 2024-09-13T23:43:32.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464c6af700 1 -- 172.21.15.167:0/1086323381 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x559f739acb00 con 0x559f738b3000 2024-09-13T23:43:32.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464c6af700 1 --2- 172.21.15.167:0/1086323381 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73c64000 0x559f73b99080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464c6af700 1 -- 172.21.15.167:0/1086323381 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x559f73919180 con 0x559f738b3000 2024-09-13T23:43:32.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464feb6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 0x559f73b98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464feb6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 0x559f73b98b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52474/0 (socket says 172.21.15.167:52474) 2024-09-13T23:43:32.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464feb6700 1 -- 172.21.15.167:0/2078294252 learned_addr learned my addr 172.21.15.167:0/2078294252 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464f6b5700 1 --2- 172.21.15.167:0/1086323381 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73c64000 0x559f73b99080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464feb6700 1 -- 172.21.15.167:0/2078294252 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f738c4000 con 0x559f738b3800 2024-09-13T23:43:32.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464feb6700 1 --2- 172.21.15.167:0/2078294252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 0x559f73b98b00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x559f73b92de0 tx=0x559f73c1cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd73cb9784c3516 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f464f6b5700 1 --2- 172.21.15.167:0/1086323381 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73c64000 0x559f73b99080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x559f74725620 tx=0x559f73c1c240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f4649eaa700 1 -- 172.21.15.167:0/2078294252 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f73957600 con 0x559f738b3800 2024-09-13T23:43:32.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f4649eaa700 1 -- 172.21.15.167:0/2078294252 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559f73c80000 con 0x559f738b3800 2024-09-13T23:43:32.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.719+0000 7f4649eaa700 1 -- 172.21.15.167:0/2078294252 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f739571e0 con 0x559f738b3800 2024-09-13T23:43:32.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f4650efcf40 1 -- 172.21.15.167:0/2078294252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 msgr2=0x559f73b98b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f4650efcf40 1 --2- 172.21.15.167:0/2078294252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 0x559f73b98b00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x559f73b92de0 tx=0x559f73c1cde0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f4650efcf40 1 -- 172.21.15.167:0/2078294252 shutdown_connections 2024-09-13T23:43:32.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f4650efcf40 1 --2- 172.21.15.167:0/2078294252 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 0x559f73b98b00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f4650efcf40 1 -- 172.21.15.167:0/2078294252 >> 172.21.15.167:0/2078294252 conn(0x559f738b3400 msgr2=0x559f739b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f4650efcf40 1 -- 172.21.15.167:0/2078294252 shutdown_connections 2024-09-13T23:43:32.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f4650efcf40 1 -- 172.21.15.167:0/2078294252 wait complete. 2024-09-13T23:43:32.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f4650efcf40 1 Processor -- start 2024-09-13T23:43:32.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f4650efcf40 1 -- start start 2024-09-13T23:43:32.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f4650efcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 0x559f73b98b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f4650efcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f738d88c0 con 0x559f738b3800 2024-09-13T23:43:32.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f464feb6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 0x559f73b98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f464feb6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 0x559f73b98b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52482/0 (socket says 172.21.15.167:52482) 2024-09-13T23:43:32.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 learned_addr learned my addr 172.21.15.167:0/162513820 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f738c4ea0 con 0x559f738b3800 2024-09-13T23:43:32.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f464feb6700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 0x559f73b98b00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x559f74724990 tx=0x559f73c1c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f46486a7700 1 -- 172.21.15.167:0/162513820 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f73957a20 con 0x559f738b3800 2024-09-13T23:43:32.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f46486a7700 1 -- 172.21.15.167:0/162513820 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559f73c80180 con 0x559f738b3800 2024-09-13T23:43:32.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559f738c4820 con 0x559f738b3800 2024-09-13T23:43:32.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.723+0000 7f46486a7700 1 -- 172.21.15.167:0/162513820 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f73957600 con 0x559f738b3800 2024-09-13T23:43:32.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f738c49c0 con 0x559f738b3800 2024-09-13T23:43:32.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f46486a7700 1 -- 172.21.15.167:0/162513820 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x559f739acdc0 con 0x559f738b3800 2024-09-13T23:43:32.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f4650efcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f46486a7700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73c64c00 0x559f73b99600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f46486a7700 1 -- 172.21.15.167:0/162513820 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x559f73919340 con 0x559f738b3800 2024-09-13T23:43:32.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f464eeb4700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73c64c00 0x559f73b99600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f4650efcf40 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559f73c65000 0x559f73b99b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65400 con 0x559f73c65000 2024-09-13T23:43:32.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f464feb6700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559f73c65000 0x559f73b99b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f464eeb4700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73c64c00 0x559f73b99600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x559f73b93920 tx=0x559f739b7800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f464feb6700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559f73c65000 0x559f73b99b80 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f4650efcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.727+0000 7f4650efcf40 20 realm 2024-09-13T23:43:32.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65000 2024-09-13T23:43:32.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:32.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65c00 con 0x559f73c65000 2024-09-13T23:43:32.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559f73c65400 0x559f73b9a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65400 2024-09-13T23:43:32.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f464f6b5700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559f73c65400 0x559f73b9a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f464f6b5700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559f73c65400 0x559f73b9a100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:32.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65c00 con 0x559f73c65400 2024-09-13T23:43:32.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:32.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 20 searching for the correct realm 2024-09-13T23:43:32.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559f73d24000 0x559f73b9a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d24400 con 0x559f73d24000 2024-09-13T23:43:32.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.731+0000 7f464eeb4700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559f73d24000 0x559f73b9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.735+0000 7f464eeb4700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559f73d24000 0x559f73b9a680 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.735+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.735+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d24400 con 0x559f73c65400 2024-09-13T23:43:32.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.735+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.735+0000 7f464f6b5700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559f73d24400 0x559f73b9ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.735+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24400 2024-09-13T23:43:32.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.735+0000 7f464feb6700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559f73d24400 0x559f73b9ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.735+0000 7f464feb6700 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559f73d24400 0x559f73b9ac00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.735+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.735+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65000 2024-09-13T23:43:32.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.735+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.735+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65000 2024-09-13T23:43:32.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24000 2024-09-13T23:43:32.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65000 2024-09-13T23:43:32.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65400 2024-09-13T23:43:32.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65400 2024-09-13T23:43:32.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65000 2024-09-13T23:43:32.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24400 2024-09-13T23:43:32.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.739+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65000 2024-09-13T23:43:32.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24400 2024-09-13T23:43:32.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65400 2024-09-13T23:43:32.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24000 2024-09-13T23:43:32.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65000 2024-09-13T23:43:32.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65400 2024-09-13T23:43:32.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24000 2024-09-13T23:43:32.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.743+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65000 2024-09-13T23:43:32.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65000 2024-09-13T23:43:32.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65000 2024-09-13T23:43:32.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24000 2024-09-13T23:43:32.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24400 2024-09-13T23:43:32.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65000 2024-09-13T23:43:32.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24400 2024-09-13T23:43:32.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.747+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24000 2024-09-13T23:43:32.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.751+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.751+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24400 2024-09-13T23:43:32.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.751+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.751+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65400 2024-09-13T23:43:32.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.751+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.751+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65400 2024-09-13T23:43:32.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.751+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.751+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24000 2024-09-13T23:43:32.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.751+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.751+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65400 2024-09-13T23:43:32.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.751+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.751+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73c65400 2024-09-13T23:43:32.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:32.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:32.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:32.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:32.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d24800 con 0x559f73c65000 2024-09-13T23:43:32.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d24c00 con 0x559f73c65400 2024-09-13T23:43:32.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:32.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25000 con 0x559f73c65400 2024-09-13T23:43:32.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:32.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 zone default found 2024-09-13T23:43:32.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 4 Realm: () 2024-09-13T23:43:32.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:32.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:32.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:32.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 zonegroup default 2024-09-13T23:43:32.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25400 con 0x559f73d24400 2024-09-13T23:43:32.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25800 con 0x559f73d24000 2024-09-13T23:43:32.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.755+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25c00 con 0x559f73d24000 2024-09-13T23:43:32.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.759+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.759+0000 7f4650efcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:32.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.759+0000 7f4650efcf40 20 started sync module instance, tier type = 2024-09-13T23:43:32.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.759+0000 7f4650efcf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:32.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.759+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25800 con 0x559f73c65400 2024-09-13T23:43:32.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.759+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.759+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94143329663744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25c00 con 0x559f73c65400 2024-09-13T23:43:32.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.759+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d24c00 con 0x559f73c65000 2024-09-13T23:43:32.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.763+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.763+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94143329664896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25000 con 0x559f73c65000 2024-09-13T23:43:32.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.763+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94143329663744] v33'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.763+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25800 con 0x559f73c65400 2024-09-13T23:43:32.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.767+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94143329664896] v33'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.767+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.767+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94143329666048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65c00 con 0x559f73c65400 2024-09-13T23:43:32.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.767+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65800 con 0x559f73d24400 2024-09-13T23:43:32.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.767+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94143329666048] v33'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.771+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.771+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94143329667200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d24800 con 0x559f73d24400 2024-09-13T23:43:32.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.771+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25400 con 0x559f73c65400 2024-09-13T23:43:32.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.771+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94143329667200] v33'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.771+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.771+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94143329668352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d24c00 con 0x559f73c65400 2024-09-13T23:43:32.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.771+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25000 con 0x559f73d24000 2024-09-13T23:43:32.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.775+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.775+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94143329669504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d32000 con 0x559f73d24000 2024-09-13T23:43:32.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.775+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d32400 con 0x559f73d24000 2024-09-13T23:43:32.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.775+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94143329668352] v33'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.775+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94143329669504] v33'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.775+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.775+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94143329670656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d32800 con 0x559f73d24000 2024-09-13T23:43:32.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.775+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d32c00 con 0x559f73d24400 2024-09-13T23:43:32.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'126 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94143329671808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d33000 con 0x559f73d24400 2024-09-13T23:43:32.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f4650efcf40 20 add_watcher() i=0 2024-09-13T23:43:32.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f4650efcf40 20 add_watcher() i=1 2024-09-13T23:43:32.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f4650efcf40 20 add_watcher() i=2 2024-09-13T23:43:32.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f4650efcf40 20 add_watcher() i=3 2024-09-13T23:43:32.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f4650efcf40 20 add_watcher() i=4 2024-09-13T23:43:32.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f4650efcf40 20 add_watcher() i=5 2024-09-13T23:43:32.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94143329670656] v33'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f4650efcf40 20 add_watcher() i=6 2024-09-13T23:43:32.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94143329671808] v33'127 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f4650efcf40 20 add_watcher() i=7 2024-09-13T23:43:32.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f4650efcf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:32.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d33400 con 0x559f73c65000 2024-09-13T23:43:32.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.779+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94143331401728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d33800 con 0x559f73c65000 2024-09-13T23:43:32.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94143331401728] v33'121 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f4636683700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:32.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f4650efcf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:32.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f4650efcf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:32.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f4650efcf40 1 Processor -- start 2024-09-13T23:43:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f4650efcf40 1 -- start start 2024-09-13T23:43:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f4650efcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 0x559f73b9b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f4650efcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f739403c0 con 0x559f73d32c00 2024-09-13T23:43:32.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f464eeb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 0x559f73b9b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f464eeb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 0x559f73b9b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52490/0 (socket says 172.21.15.167:52490) 2024-09-13T23:43:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f464eeb4700 1 -- 172.21.15.167:0/1120552105 learned_addr learned my addr 172.21.15.167:0/1120552105 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f464eeb4700 1 -- 172.21.15.167:0/1120552105 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f738c4b60 con 0x559f73d32c00 2024-09-13T23:43:32.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.783+0000 7f464eeb4700 1 --2- 172.21.15.167:0/1120552105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 0x559f73b9b180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x559f73cc9bf0 tx=0x559f739b7860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc17a18db6ae60f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4635e82700 1 -- 172.21.15.167:0/1120552105 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f73957760 con 0x559f73d32c00 2024-09-13T23:43:32.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4635e82700 1 -- 172.21.15.167:0/1120552105 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559f73c80600 con 0x559f73d32c00 2024-09-13T23:43:32.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4635e82700 1 -- 172.21.15.167:0/1120552105 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f73d4a000 con 0x559f73d32c00 2024-09-13T23:43:32.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 -- 172.21.15.167:0/1120552105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 msgr2=0x559f73b9b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1120552105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 0x559f73b9b180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x559f73cc9bf0 tx=0x559f739b7860 comp rx=0 tx=0).stop 2024-09-13T23:43:32.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 -- 172.21.15.167:0/1120552105 shutdown_connections 2024-09-13T23:43:32.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1120552105 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 0x559f73b9b180 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 -- 172.21.15.167:0/1120552105 >> 172.21.15.167:0/1120552105 conn(0x559f73d33c00 msgr2=0x559f739b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 -- 172.21.15.167:0/1120552105 shutdown_connections 2024-09-13T23:43:32.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 -- 172.21.15.167:0/1120552105 wait complete. 2024-09-13T23:43:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 Processor -- start 2024-09-13T23:43:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 -- start start 2024-09-13T23:43:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 0x559f73b9b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f73940500 con 0x559f73d32c00 2024-09-13T23:43:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f464eeb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 0x559f73b9b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f464eeb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 0x559f73b9b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52502/0 (socket says 172.21.15.167:52502) 2024-09-13T23:43:32.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f464eeb4700 1 -- 172.21.15.167:0/1768747117 learned_addr learned my addr 172.21.15.167:0/1768747117 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f464eeb4700 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f738c4d00 con 0x559f73d32c00 2024-09-13T23:43:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f464eeb4700 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 0x559f73b9b180 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x559f73d401e0 tx=0x559f73d42300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f463467f700 1 -- 172.21.15.167:0/1768747117 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f73d4a160 con 0x559f73d32c00 2024-09-13T23:43:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f463467f700 1 -- 172.21.15.167:0/1768747117 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559f73c80300 con 0x559f73d32c00 2024-09-13T23:43:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f463467f700 1 -- 172.21.15.167:0/1768747117 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559f73d4a2c0 con 0x559f73d32c00 2024-09-13T23:43:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559f738c4340 con 0x559f73d32c00 2024-09-13T23:43:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.787+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f738c5d40 con 0x559f73d32c00 2024-09-13T23:43:32.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.791+0000 7f463467f700 1 -- 172.21.15.167:0/1768747117 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x559f739adb80 con 0x559f73d32c00 2024-09-13T23:43:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.791+0000 7f463467f700 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73dbe000 0x559f73dc4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.791+0000 7f463467f700 1 -- 172.21.15.167:0/1768747117 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x559f73919500 con 0x559f73d32c00 2024-09-13T23:43:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.791+0000 7f464feb6700 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73dbe000 0x559f73dc4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.791+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559f73d33000 0x559f73b9b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.791+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d33400 con 0x559f73d33000 2024-09-13T23:43:32.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.791+0000 7f464f6b5700 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559f73d33000 0x559f73b9b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.791+0000 7f464feb6700 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73dbe000 0x559f73dc4000 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x559f73cc96b0 tx=0x559f73c1d7a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.791+0000 7f464f6b5700 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559f73d33000 0x559f73b9b700 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.795+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'27 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.795+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559f73d32000 0x559f73dc4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.795+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d32400 con 0x559f73d32000 2024-09-13T23:43:32.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.795+0000 7f464eeb4700 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559f73d32000 0x559f73dc4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.795+0000 7f464eeb4700 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559f73d32000 0x559f73dc4580 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.795+0000 7f464eeb4700 1 -- 172.21.15.167:0/1768747117 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'15 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d32000 2024-09-13T23:43:32.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.795+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559f73d24800 0x559f73dc4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.795+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25000 con 0x559f73d24800 2024-09-13T23:43:32.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.799+0000 7f464feb6700 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559f73d24800 0x559f73dc4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.799+0000 7f464feb6700 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559f73d24800 0x559f73dc4b00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.799+0000 7f464feb6700 1 -- 172.21.15.167:0/1768747117 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'15 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24800 2024-09-13T23:43:32.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.799+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559f73c65800 0x559f73dc5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:32.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.799+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbe400 con 0x559f73c65800 2024-09-13T23:43:32.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.799+0000 7f464f6b5700 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559f73c65800 0x559f73dc5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:32.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.799+0000 7f464f6b5700 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559f73c65800 0x559f73dc5080 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:32.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.803+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'122 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65800 2024-09-13T23:43:32.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.803+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbe800 con 0x559f73d33000 2024-09-13T23:43:32.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.807+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'35 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.807+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbec00 con 0x559f73d24800 2024-09-13T23:43:32.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.807+0000 7f464feb6700 1 -- 172.21.15.167:0/1768747117 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'26 uv10 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24800 2024-09-13T23:43:32.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.807+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbf000 con 0x559f73d33000 2024-09-13T23:43:32.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.811+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'17 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.811+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbf400 con 0x559f73c65800 2024-09-13T23:43:32.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.815+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'24 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65800 2024-09-13T23:43:32.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.815+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbf800 con 0x559f73d33000 2024-09-13T23:43:32.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.815+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.815+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbfc00 con 0x559f73d33000 2024-09-13T23:43:32.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.819+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'22 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.819+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d33400 con 0x559f73d33000 2024-09-13T23:43:32.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.823+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'36 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.823+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25c00 con 0x559f73d24800 2024-09-13T23:43:32.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.823+0000 7f464feb6700 1 -- 172.21.15.167:0/1768747117 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'17 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24800 2024-09-13T23:43:32.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.823+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25800 con 0x559f73d33000 2024-09-13T23:43:32.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.827+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.827+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25400 con 0x559f73c65800 2024-09-13T23:43:32.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.831+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'29 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65800 2024-09-13T23:43:32.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.831+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d24c00 con 0x559f73c65800 2024-09-13T23:43:32.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.831+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'123 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65800 2024-09-13T23:43:32.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.831+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65c00 con 0x559f73d33000 2024-09-13T23:43:32.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.835+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'29 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.835+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d33400 con 0x559f73d33000 2024-09-13T23:43:32.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.839+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'37 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.839+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbe400 con 0x559f73d24800 2024-09-13T23:43:32.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.839+0000 7f464feb6700 1 -- 172.21.15.167:0/1768747117 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'15 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24800 2024-09-13T23:43:32.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.839+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbe800 con 0x559f73d33000 2024-09-13T23:43:32.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.843+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'23 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.843+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbf000 con 0x559f73c65800 2024-09-13T23:43:32.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.847+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'124 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65800 2024-09-13T23:43:32.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.847+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbf400 con 0x559f73d33000 2024-09-13T23:43:32.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.847+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'30 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.847+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbf800 con 0x559f73d33000 2024-09-13T23:43:32.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.851+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'25 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.851+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbfc00 con 0x559f73d24800 2024-09-13T23:43:32.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.855+0000 7f464feb6700 1 -- 172.21.15.167:0/1768747117 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24800 2024-09-13T23:43:32.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.855+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbec00 con 0x559f73d33000 2024-09-13T23:43:32.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.859+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.859+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbe400 con 0x559f73d24800 2024-09-13T23:43:32.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.859+0000 7f464feb6700 1 -- 172.21.15.167:0/1768747117 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'11 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24800 2024-09-13T23:43:32.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.859+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbfc00 con 0x559f73d24800 2024-09-13T23:43:32.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.863+0000 7f464feb6700 1 -- 172.21.15.167:0/1768747117 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'27 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24800 2024-09-13T23:43:32.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.863+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbe800 con 0x559f73d24800 2024-09-13T23:43:32.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.863+0000 7f464feb6700 1 -- 172.21.15.167:0/1768747117 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24800 2024-09-13T23:43:32.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.867+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbf000 con 0x559f73d33000 2024-09-13T23:43:32.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.867+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'15 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.867+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbf400 con 0x559f73c65800 2024-09-13T23:43:32.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.871+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'25 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65800 2024-09-13T23:43:32.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.871+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d33800 con 0x559f73d33000 2024-09-13T23:43:32.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.875+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'31 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73d33000 2024-09-13T23:43:32.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.875+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d33400 con 0x559f73c65800 2024-09-13T23:43:32.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.875+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'30 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65800 2024-09-13T23:43:32.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.875+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73c65c00 con 0x559f73d32000 2024-09-13T23:43:32.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.879+0000 7f464eeb4700 1 -- 172.21.15.167:0/1768747117 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'21 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d32000 2024-09-13T23:43:32.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.879+0000 7f4631e7a700 20 reqs_thread_entry: start 2024-09-13T23:43:32.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f4650efcf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:32.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f4650efcf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:32.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f462f654700 20 reqs_thread_entry: start 2024-09-13T23:43:32.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f462fe76700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:03 2024 2024-09-13T23:43:32.891 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:32.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f4650efcf40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-13T23:43:32.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f4650efcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:32.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25800 con 0x559f73d24400 2024-09-13T23:43:32.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f4650efcf40 20 rados_obj.operate() r=0 bl.length=346 2024-09-13T23:43:32.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f4650efcf40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-09-13T23:43:32.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f4650efcf40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:43:32.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f4650efcf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:32.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25400 con 0x559f73c65000 2024-09-13T23:43:32.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(63 fud.buckets [call out=15b] v0'0 uv2 ondisk = 0) v8 ==== 155+0+15 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f4650efcf40 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-09-13T23:43:32.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.883+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d24c00 con 0x559f73d24000 2024-09-13T23:43:32.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.887+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(64 p5YnriCv1nAtykxBrupQ [delete] v33'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.887+0000 7f4650efcf40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-13T23:43:32.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.887+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [notify cookie 94143331405184 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e70000 con 0x559f73c65400 2024-09-13T23:43:32.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.887+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94143329666048 notify 141733920789 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x559f73c1b380 con 0x559f73c65400 2024-09-13T23:43:32.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.887+0000 7f4649eaa700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920789 cookie 94143329666048 notifier 4566 bl.length()=170 2024-09-13T23:43:32.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.887+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 notify.2 [notify cookie 94143331405184 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.887+0000 7f4649eaa700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e76000 con 0x559f73c65400 2024-09-13T23:43:32.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.887+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(66 notify.2 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.887+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94143331405184 notify 141733920789 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x559f73c1b520 con 0x559f73c65400 2024-09-13T23:43:32.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.887+0000 7f4650efcf40 10 removing email index: 2024-09-13T23:43:32.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.887+0000 7f4650efcf40 10 removing user buckets index 2024-09-13T23:43:32.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.887+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e70400 con 0x559f73c65000 2024-09-13T23:43:32.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.891+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(67 fud.buckets [delete] v33'4 uv4 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.891+0000 7f4650efcf40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-09-13T23:43:32.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.891+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94143331405184 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e70800 con 0x559f73d24000 2024-09-13T23:43:32.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.891+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== watch-notify(notify (1) cookie 94143329670656 notify 141733920774 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x559f73d8a9c0 con 0x559f73d24000 2024-09-13T23:43:32.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.891+0000 7f4649eaa700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920774 cookie 94143329670656 notifier 4566 bl.length()=160 2024-09-13T23:43:32.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.891+0000 7f4649eaa700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e76400 con 0x559f73d24000 2024-09-13T23:43:32.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.891+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(68 notify.6 [notify cookie 94143331405184 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.891+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== watch-notify(notify_complete (2) cookie 94143331405184 notify 141733920774 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x559f73d8ab60 con 0x559f73d24000 2024-09-13T23:43:32.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.895+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.895+0000 7f4650efcf40 10 removing user index: fud 2024-09-13T23:43:32.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.895+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e70c00 con 0x559f73d24400 2024-09-13T23:43:32.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.895+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(70 fud [call,call,delete] v33'2 uv2 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.895+0000 7f4650efcf40 10 removing default.rgw.meta+users.uid+fud from cache 2024-09-13T23:43:32.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.895+0000 7f4650efcf40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-09-13T23:43:32.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.895+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [notify cookie 94143331405184 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e71000 con 0x559f73d24000 2024-09-13T23:43:32.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.895+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== watch-notify(notify (1) cookie 94143329669504 notify 141733920775 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x559f73d8ab60 con 0x559f73d24000 2024-09-13T23:43:32.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.895+0000 7f46496a9700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920775 cookie 94143329669504 notifier 4566 bl.length()=152 2024-09-13T23:43:32.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.895+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(71 notify.5 [notify cookie 94143331405184 out=8b] v0'0 uv126 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.895+0000 7f46496a9700 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73dbe400 con 0x559f73d24000 2024-09-13T23:43:32.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.899+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(72 notify.5 [notify-ack] v0'0 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.899+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== watch-notify(notify_complete (2) cookie 94143331405184 notify 141733920775 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x559f73d8ad00 con 0x559f73d24000 2024-09-13T23:43:32.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.899+0000 7f4636683700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:32.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.899+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94143331401728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e71400 con 0x559f73c65000 2024-09-13T23:43:32.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.903+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94143331401728] v33'125 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.903+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94143329663744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e71800 con 0x559f73c65400 2024-09-13T23:43:32.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.903+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94143329663744] v33'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.903+0000 7f4650efcf40 20 remove_watcher() i=0 2024-09-13T23:43:32.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.903+0000 7f4650efcf40 2 removed watcher, disabling cache 2024-09-13T23:43:32.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.903+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94143329664896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e71c00 con 0x559f73c65000 2024-09-13T23:43:32.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.907+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94143329664896] v33'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73c65000 2024-09-13T23:43:32.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.907+0000 7f4650efcf40 20 remove_watcher() i=1 2024-09-13T23:43:32.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.907+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94143329666048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e70000 con 0x559f73c65400 2024-09-13T23:43:32.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.907+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94143329666048] v33'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.907+0000 7f4650efcf40 20 remove_watcher() i=2 2024-09-13T23:43:32.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.907+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94143329667200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e71800 con 0x559f73d24400 2024-09-13T23:43:32.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.911+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94143329667200] v33'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.911+0000 7f4650efcf40 20 remove_watcher() i=3 2024-09-13T23:43:32.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.911+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94143329668352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e70400 con 0x559f73c65400 2024-09-13T23:43:32.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.915+0000 7f464f6b5700 1 -- 172.21.15.167:0/162513820 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94143329668352] v33'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b200 con 0x559f73c65400 2024-09-13T23:43:32.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.915+0000 7f4650efcf40 20 remove_watcher() i=4 2024-09-13T23:43:32.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.915+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94143329669504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73e70c00 con 0x559f73d24000 2024-09-13T23:43:32.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.915+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94143329669504] v33'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.915+0000 7f4650efcf40 20 remove_watcher() i=5 2024-09-13T23:43:32.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.915+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94143329670656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25800 con 0x559f73d24000 2024-09-13T23:43:32.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94143329670656] v33'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0b440 con 0x559f73d24000 2024-09-13T23:43:32.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 20 remove_watcher() i=6 2024-09-13T23:43:32.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94143329671808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559f73d25400 con 0x559f73d24400 2024-09-13T23:43:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f464feb6700 1 -- 172.21.15.167:0/162513820 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94143329671808] v33'128 uv126 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559f73d0ab40 con 0x559f73d24400 2024-09-13T23:43:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 20 remove_watcher() i=7 2024-09-13T23:43:32.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559f73c65400 msgr2=0x559f73b9a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559f73c65400 0x559f73b9a100 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559f73c65000 msgr2=0x559f73b99b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559f73c65000 0x559f73b99b80 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559f73d24400 msgr2=0x559f73b9ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559f73d24400 0x559f73b9ac00 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559f73d24000 msgr2=0x559f73b9a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559f73d24000 0x559f73b9a680 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73c64c00 msgr2=0x559f73b99600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73c64c00 0x559f73b99600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x559f73b93920 tx=0x559f739b7800 comp rx=0 tx=0).stop 2024-09-13T23:43:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 msgr2=0x559f73b98b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 --2- 172.21.15.167:0/162513820 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3800 0x559f73b98b00 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x559f74724990 tx=0x559f73c1c120 comp rx=0 tx=0).stop 2024-09-13T23:43:32.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 reap_dead start 2024-09-13T23:43:32.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f464eeb4700 1 -- 172.21.15.167:0/162513820 reap_dead start 2024-09-13T23:43:32.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 shutdown_connections 2024-09-13T23:43:32.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.919+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 >> 172.21.15.167:0/162513820 conn(0x559f738b3400 msgr2=0x559f739b3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 shutdown_connections 2024-09-13T23:43:32.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/162513820 wait complete. 2024-09-13T23:43:32.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559f73d33000 msgr2=0x559f73b9b700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559f73d33000 0x559f73b9b700 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559f73c65800 msgr2=0x559f73dc5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559f73c65800 0x559f73dc5080 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559f73d32000 msgr2=0x559f73dc4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559f73d32000 0x559f73dc4580 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559f73d24800 msgr2=0x559f73dc4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559f73d24800 0x559f73dc4b00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73dbe000 msgr2=0x559f73dc4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73dbe000 0x559f73dc4000 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x559f73cc96b0 tx=0x559f73c1d7a0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 msgr2=0x559f73b9b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 0x559f73b9b180 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x559f73d401e0 tx=0x559f73d42300 comp rx=0 tx=0).stop 2024-09-13T23:43:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f464f6b5700 1 -- 172.21.15.167:0/1768747117 reap_dead start 2024-09-13T23:43:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 shutdown_connections 2024-09-13T23:43:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1768747117 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f73d32c00 0x559f73b9b180 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 >> 172.21.15.167:0/1768747117 conn(0x559f73d33c00 msgr2=0x559f739b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 shutdown_connections 2024-09-13T23:43:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1768747117 wait complete. 2024-09-13T23:43:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1086323381 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73c64000 msgr2=0x559f73b99080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1086323381 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73c64000 0x559f73b99080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x559f74725620 tx=0x559f73c1c240 comp rx=0 tx=0).stop 2024-09-13T23:43:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1086323381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 msgr2=0x559f73b98580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1086323381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 0x559f73b98580 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x559f747240c0 tx=0x559f739b7a40 comp rx=0 tx=0).stop 2024-09-13T23:43:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1086323381 shutdown_connections 2024-09-13T23:43:32.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1086323381 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559f73c64000 0x559f73b99080 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 --2- 172.21.15.167:0/1086323381 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559f738b3000 0x559f73b98580 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:32.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1086323381 >> 172.21.15.167:0/1086323381 conn(0x559f738b2c00 msgr2=0x559f739b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:32.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1086323381 shutdown_connections 2024-09-13T23:43:32.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:32.923+0000 7f4650efcf40 1 -- 172.21.15.167:0/1086323381 wait complete. 2024-09-13T23:43:32.938 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-09-13T23:43:32.938 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-09-13T23:43:32.938 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --tenant testx --uid tenanteduser --display-name tenanted-user --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-09-13T23:43:33.031 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:33.031 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:33.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45aee6df40 1 Processor -- start 2024-09-13T23:43:33.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45aee6df40 1 -- start start 2024-09-13T23:43:33.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45aee6df40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ddf1869400 0x55ddf1b4e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45aee6df40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ddf192d500 con 0x55ddf1869000 2024-09-13T23:43:33.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45aee6df40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ddf18f6a00 con 0x55ddf1869400 2024-09-13T23:43:33.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45ace25700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ddf1869400 0x55ddf1b4e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45ace25700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ddf1869400 0x55ddf1b4e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52514/0 (socket says 172.21.15.167:52514) 2024-09-13T23:43:33.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45ad626700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55ddf1869000 0x55ddf1968800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45518/0 (socket says 172.21.15.167:45518) 2024-09-13T23:43:33.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45ace25700 1 -- 172.21.15.167:0/2568673852 learned_addr learned my addr 172.21.15.167:0/2568673852 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45ace25700 1 -- 172.21.15.167:0/2568673852 >> v1:172.21.15.12:6789/0 conn(0x55ddf1869000 legacy=0x55ddf1968800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45ace25700 1 -- 172.21.15.167:0/2568673852 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf187bd40 con 0x55ddf1869400 2024-09-13T23:43:33.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45ace25700 1 --2- 172.21.15.167:0/2568673852 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869400 0x55ddf1b4e580 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x55ddf1b49920 tx=0x55ddf196d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cb54ce792c458c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45ac624700 1 -- 172.21.15.167:0/2568673852 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf190d4a0 con 0x55ddf1869400 2024-09-13T23:43:33.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45ac624700 1 -- 172.21.15.167:0/2568673852 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddf192d800 con 0x55ddf1869400 2024-09-13T23:43:33.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.023+0000 7f45ac624700 1 -- 172.21.15.167:0/2568673852 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf190d600 con 0x55ddf1869400 2024-09-13T23:43:33.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45aee6df40 1 -- 172.21.15.167:0/2568673852 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869400 msgr2=0x55ddf1b4e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2568673852 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869400 0x55ddf1b4e580 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x55ddf1b49920 tx=0x55ddf196d920 comp rx=0 tx=0).stop 2024-09-13T23:43:33.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45aee6df40 1 -- 172.21.15.167:0/2568673852 shutdown_connections 2024-09-13T23:43:33.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2568673852 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869400 0x55ddf1b4e580 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45aee6df40 1 -- 172.21.15.167:0/2568673852 >> 172.21.15.167:0/2568673852 conn(0x55ddf1868c00 msgr2=0x55ddf1968000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45aee6df40 1 -- 172.21.15.167:0/2568673852 shutdown_connections 2024-09-13T23:43:33.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45aee6df40 1 -- 172.21.15.167:0/2568673852 wait complete. 2024-09-13T23:43:33.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45aee6df40 1 Processor -- start 2024-09-13T23:43:33.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45aee6df40 1 -- start start 2024-09-13T23:43:33.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45aee6df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 0x55ddf1b4e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45aee6df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf18f6500 con 0x55ddf1869000 2024-09-13T23:43:33.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45ad626700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 0x55ddf1b4e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45ad626700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 0x55ddf1b4e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52528/0 (socket says 172.21.15.167:52528) 2024-09-13T23:43:33.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45ad626700 1 -- 172.21.15.167:0/1602692386 learned_addr learned my addr 172.21.15.167:0/1602692386 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.027+0000 7f45ad626700 1 -- 172.21.15.167:0/1602692386 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf187bba0 con 0x55ddf1869000 2024-09-13T23:43:33.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45ad626700 1 --2- 172.21.15.167:0/1602692386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 0x55ddf1b4e580 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x55ddf19c5e90 tx=0x55ddf196db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48297fec18ad72f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45abe23700 1 -- 172.21.15.167:0/1602692386 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf190d4a0 con 0x55ddf1869000 2024-09-13T23:43:33.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45abe23700 1 -- 172.21.15.167:0/1602692386 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddf192db00 con 0x55ddf1869000 2024-09-13T23:43:33.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45abe23700 1 -- 172.21.15.167:0/1602692386 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf190d600 con 0x55ddf1869000 2024-09-13T23:43:33.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45aee6df40 1 -- 172.21.15.167:0/1602692386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 msgr2=0x55ddf1b4e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45aee6df40 1 --2- 172.21.15.167:0/1602692386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 0x55ddf1b4e580 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x55ddf19c5e90 tx=0x55ddf196db60 comp rx=0 tx=0).stop 2024-09-13T23:43:33.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45aee6df40 1 -- 172.21.15.167:0/1602692386 shutdown_connections 2024-09-13T23:43:33.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45aee6df40 1 --2- 172.21.15.167:0/1602692386 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 0x55ddf1b4e580 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45aee6df40 1 -- 172.21.15.167:0/1602692386 >> 172.21.15.167:0/1602692386 conn(0x55ddf1868c00 msgr2=0x55ddf1969000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45aee6df40 1 -- 172.21.15.167:0/1602692386 shutdown_connections 2024-09-13T23:43:33.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45aee6df40 1 -- 172.21.15.167:0/1602692386 wait complete. 2024-09-13T23:43:33.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45aee6df40 1 Processor -- start 2024-09-13T23:43:33.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45aee6df40 1 -- start start 2024-09-13T23:43:33.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45aee6df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 0x55ddf1b4e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45aee6df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf18f63c0 con 0x55ddf1869000 2024-09-13T23:43:33.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45ad626700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 0x55ddf1b4e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45ad626700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 0x55ddf1b4e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52532/0 (socket says 172.21.15.167:52532) 2024-09-13T23:43:33.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.031+0000 7f45ad626700 1 -- 172.21.15.167:0/2559469617 learned_addr learned my addr 172.21.15.167:0/2559469617 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45ad626700 1 -- 172.21.15.167:0/2559469617 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf187ba00 con 0x55ddf1869000 2024-09-13T23:43:33.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45ad626700 1 --2- 172.21.15.167:0/2559469617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 0x55ddf1b4e580 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55ddf26d80c0 tx=0x55ddf196da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45aa620700 1 -- 172.21.15.167:0/2559469617 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf190d8c0 con 0x55ddf1869000 2024-09-13T23:43:33.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45aa620700 1 -- 172.21.15.167:0/2559469617 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddf192dc80 con 0x55ddf1869000 2024-09-13T23:43:33.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45aee6df40 1 -- 172.21.15.167:0/2559469617 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ddf187b860 con 0x55ddf1869000 2024-09-13T23:43:33.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45aa620700 1 -- 172.21.15.167:0/2559469617 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf190d4a0 con 0x55ddf1869000 2024-09-13T23:43:33.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45aee6df40 1 -- 172.21.15.167:0/2559469617 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddf187b6c0 con 0x55ddf1869000 2024-09-13T23:43:33.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45aee6df40 1 Processor -- start 2024-09-13T23:43:33.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45aa620700 1 -- 172.21.15.167:0/2559469617 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ddf1962b00 con 0x55ddf1869000 2024-09-13T23:43:33.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45aee6df40 1 -- start start 2024-09-13T23:43:33.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45aa620700 1 --2- 172.21.15.167:0/2559469617 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1c10000 0x55ddf1b4f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45aa620700 1 -- 172.21.15.167:0/2559469617 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ddf18cf180 con 0x55ddf1869000 2024-09-13T23:43:33.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45aee6df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 0x55ddf1b4eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45aee6df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf188e640 con 0x55ddf1869800 2024-09-13T23:43:33.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45ade27700 1 --2- 172.21.15.167:0/2559469617 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1c10000 0x55ddf1b4f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45ad626700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 0x55ddf1b4eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45ad626700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 0x55ddf1b4eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52534/0 (socket says 172.21.15.167:52534) 2024-09-13T23:43:33.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.035+0000 7f45ad626700 1 -- 172.21.15.167:0/2866605770 learned_addr learned my addr 172.21.15.167:0/2866605770 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45ade27700 1 --2- 172.21.15.167:0/2559469617 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1c10000 0x55ddf1b4f080 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55ddf26d8ed0 tx=0x55ddf1bccde0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45ad626700 1 -- 172.21.15.167:0/2866605770 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf187a000 con 0x55ddf1869800 2024-09-13T23:43:33.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45ad626700 1 --2- 172.21.15.167:0/2866605770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 0x55ddf1b4eb00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55ddf26d9170 tx=0x55ddf1bcc240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3e019144bafbb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45a7e1b700 1 -- 172.21.15.167:0/2866605770 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf190d600 con 0x55ddf1869800 2024-09-13T23:43:33.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45a7e1b700 1 -- 172.21.15.167:0/2866605770 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddf192dc80 con 0x55ddf1869800 2024-09-13T23:43:33.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45a7e1b700 1 -- 172.21.15.167:0/2866605770 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf190da20 con 0x55ddf1869800 2024-09-13T23:43:33.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45aee6df40 1 -- 172.21.15.167:0/2866605770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 msgr2=0x55ddf1b4eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2866605770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 0x55ddf1b4eb00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x55ddf26d9170 tx=0x55ddf1bcc240 comp rx=0 tx=0).stop 2024-09-13T23:43:33.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45aee6df40 1 -- 172.21.15.167:0/2866605770 shutdown_connections 2024-09-13T23:43:33.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2866605770 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 0x55ddf1b4eb00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45aee6df40 1 -- 172.21.15.167:0/2866605770 >> 172.21.15.167:0/2866605770 conn(0x55ddf1869400 msgr2=0x55ddf1969800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45aee6df40 1 -- 172.21.15.167:0/2866605770 shutdown_connections 2024-09-13T23:43:33.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45aee6df40 1 -- 172.21.15.167:0/2866605770 wait complete. 2024-09-13T23:43:33.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45aee6df40 1 Processor -- start 2024-09-13T23:43:33.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45aee6df40 1 -- start start 2024-09-13T23:43:33.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45aee6df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 0x55ddf1b4eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45aee6df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf188e8c0 con 0x55ddf1869800 2024-09-13T23:43:33.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45ad626700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 0x55ddf1b4eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45ad626700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 0x55ddf1b4eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52536/0 (socket says 172.21.15.167:52536) 2024-09-13T23:43:33.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 learned_addr learned my addr 172.21.15.167:0/2389959210 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.039+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf187aea0 con 0x55ddf1869800 2024-09-13T23:43:33.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45ad626700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 0x55ddf1b4eb00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55ddf26d83c0 tx=0x55ddf1bccfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45a6618700 1 -- 172.21.15.167:0/2389959210 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf190d600 con 0x55ddf1869800 2024-09-13T23:43:33.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45a6618700 1 -- 172.21.15.167:0/2389959210 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddf1c30000 con 0x55ddf1869800 2024-09-13T23:43:33.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ddf187a820 con 0x55ddf1869800 2024-09-13T23:43:33.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45a6618700 1 -- 172.21.15.167:0/2389959210 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf190da20 con 0x55ddf1869800 2024-09-13T23:43:33.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddf187a9c0 con 0x55ddf1869800 2024-09-13T23:43:33.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45a6618700 1 -- 172.21.15.167:0/2389959210 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ddf1962dc0 con 0x55ddf1869800 2024-09-13T23:43:33.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45a6618700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1c11000 0x55ddf1b4f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45a6618700 1 -- 172.21.15.167:0/2389959210 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ddf18cf340 con 0x55ddf1869800 2024-09-13T23:43:33.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ddf1c11400 0x55ddf1b4fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45ace25700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1c11000 0x55ddf1b4f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c11800 con 0x55ddf1c11400 2024-09-13T23:43:33.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45ade27700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ddf1c11400 0x55ddf1b4fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45ace25700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1c11000 0x55ddf1b4f600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55ddf1b49920 tx=0x55ddf196d800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45ade27700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ddf1c11400 0x55ddf1b4fb80 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45aee6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45aee6df40 20 realm 2024-09-13T23:43:33.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.043+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c11c00 con 0x55ddf1c11400 2024-09-13T23:43:33.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:33.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c11800 con 0x55ddf1c11400 2024-09-13T23:43:33.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ddf1c11c00 0x55ddf1b50100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e000 con 0x55ddf1c11c00 2024-09-13T23:43:33.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45ad626700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ddf1c11c00 0x55ddf1b50100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45ad626700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ddf1c11c00 0x55ddf1b50100 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:33.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11c00 2024-09-13T23:43:33.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:33.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 20 searching for the correct realm 2024-09-13T23:43:33.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ddf1c9e800 0x55ddf1b50680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.047+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9ec00 con 0x55ddf1c9e800 2024-09-13T23:43:33.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ace25700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ddf1c9e800 0x55ddf1b50680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ace25700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ddf1c9e800 0x55ddf1b50680 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9ec00 con 0x55ddf1c11c00 2024-09-13T23:43:33.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ad626700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ddf1c9e000 0x55ddf1b50c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e000 2024-09-13T23:43:33.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ade27700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ddf1c9e000 0x55ddf1b50c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ade27700 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ddf1c9e000 0x55ddf1b50c00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11400 2024-09-13T23:43:33.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.051+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11400 2024-09-13T23:43:33.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e800 2024-09-13T23:43:33.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11400 2024-09-13T23:43:33.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11c00 2024-09-13T23:43:33.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11c00 2024-09-13T23:43:33.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11400 2024-09-13T23:43:33.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e000 2024-09-13T23:43:33.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.055+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11400 2024-09-13T23:43:33.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e000 2024-09-13T23:43:33.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11c00 2024-09-13T23:43:33.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e800 2024-09-13T23:43:33.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11400 2024-09-13T23:43:33.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11c00 2024-09-13T23:43:33.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e800 2024-09-13T23:43:33.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11400 2024-09-13T23:43:33.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.059+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11400 2024-09-13T23:43:33.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.063+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.063+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11400 2024-09-13T23:43:33.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.063+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.063+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e800 2024-09-13T23:43:33.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.063+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.063+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e000 2024-09-13T23:43:33.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.063+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.063+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11400 2024-09-13T23:43:33.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.063+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.063+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e000 2024-09-13T23:43:33.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.063+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.063+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e800 2024-09-13T23:43:33.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e000 2024-09-13T23:43:33.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11c00 2024-09-13T23:43:33.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11c00 2024-09-13T23:43:33.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e800 2024-09-13T23:43:33.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11c00 2024-09-13T23:43:33.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11c00 2024-09-13T23:43:33.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.067+0000 7f45aee6df40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:33.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:33.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:33.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:33.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9f000 con 0x55ddf1c11400 2024-09-13T23:43:33.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9f400 con 0x55ddf1c11c00 2024-09-13T23:43:33.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:33.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9f800 con 0x55ddf1c11c00 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 zone default found 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 4 Realm: () 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 zonegroup default 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9fc00 con 0x55ddf1c9e000 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9f000 con 0x55ddf1c9e800 2024-09-13T23:43:33.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9fc00 con 0x55ddf1c9e800 2024-09-13T23:43:33.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 started sync module instance, tier type = 2024-09-13T23:43:33.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:33.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.071+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c11800 con 0x55ddf1c11c00 2024-09-13T23:43:33.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.075+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.075+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94411731262208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9ec00 con 0x55ddf1c11c00 2024-09-13T23:43:33.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.075+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c11400 2024-09-13T23:43:33.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.075+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.075+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94411731263360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9f400 con 0x55ddf1c11400 2024-09-13T23:43:33.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.079+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9f800 con 0x55ddf1c11c00 2024-09-13T23:43:33.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.079+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94411731262208] v33'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.079+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94411731263360] v33'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.079+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.079+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94411731264512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9f000 con 0x55ddf1c11c00 2024-09-13T23:43:33.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.079+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9fc00 con 0x55ddf1c9e000 2024-09-13T23:43:33.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.079+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.083+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94411731265664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c10800 con 0x55ddf1c9e000 2024-09-13T23:43:33.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.083+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c11800 con 0x55ddf1c11c00 2024-09-13T23:43:33.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.083+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94411731264512] v33'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.083+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94411731265664] v33'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.083+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.083+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94411731266816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c10800 con 0x55ddf1c11c00 2024-09-13T23:43:33.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.083+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e800 2024-09-13T23:43:33.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.087+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.087+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94411731267968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9f400 con 0x55ddf1c9e800 2024-09-13T23:43:33.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.087+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9fc00 con 0x55ddf1c9e800 2024-09-13T23:43:33.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.087+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94411731266816] v33'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.087+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94411731267968] v33'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.087+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.087+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94411731269120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9e400 con 0x55ddf1c9e800 2024-09-13T23:43:33.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.087+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9f400 con 0x55ddf1c9e000 2024-09-13T23:43:33.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'129 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94411731270272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9fc00 con 0x55ddf1c9e000 2024-09-13T23:43:33.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45aee6df40 20 add_watcher() i=0 2024-09-13T23:43:33.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45aee6df40 20 add_watcher() i=1 2024-09-13T23:43:33.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45aee6df40 20 add_watcher() i=2 2024-09-13T23:43:33.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45aee6df40 20 add_watcher() i=3 2024-09-13T23:43:33.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45aee6df40 20 add_watcher() i=4 2024-09-13T23:43:33.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45aee6df40 20 add_watcher() i=5 2024-09-13T23:43:33.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94411731269120] v33'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45aee6df40 20 add_watcher() i=6 2024-09-13T23:43:33.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94411731270272] v33'130 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45aee6df40 20 add_watcher() i=7 2024-09-13T23:43:33.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45aee6df40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:33.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1ca8000 con 0x55ddf1c11400 2024-09-13T23:43:33.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.091+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94411732729856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1ca8400 con 0x55ddf1c11400 2024-09-13T23:43:33.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.095+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94411732729856] v33'126 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.095+0000 7f45945f4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:33.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.095+0000 7f45aee6df40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:33.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.095+0000 7f45aee6df40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:33.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.095+0000 7f45aee6df40 1 Processor -- start 2024-09-13T23:43:33.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.095+0000 7f45aee6df40 1 -- start start 2024-09-13T23:43:33.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.095+0000 7f45aee6df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 0x55ddf1b51180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.095+0000 7f45aee6df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf18f63c0 con 0x55ddf1ca8c00 2024-09-13T23:43:33.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.095+0000 7f45ace25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 0x55ddf1b51180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.095+0000 7f45ace25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 0x55ddf1b51180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52546/0 (socket says 172.21.15.167:52546) 2024-09-13T23:43:33.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.095+0000 7f45ace25700 1 -- 172.21.15.167:0/2665389452 learned_addr learned my addr 172.21.15.167:0/2665389452 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.095+0000 7f45ace25700 1 -- 172.21.15.167:0/2665389452 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf187ab60 con 0x55ddf1ca8c00 2024-09-13T23:43:33.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45ace25700 1 --2- 172.21.15.167:0/2665389452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 0x55ddf1b51180 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x55ddf1c51560 tx=0x55ddf196d860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebfa97e97284cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f4593df3700 1 -- 172.21.15.167:0/2665389452 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf190d760 con 0x55ddf1ca8c00 2024-09-13T23:43:33.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f4593df3700 1 -- 172.21.15.167:0/2665389452 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddf1c30900 con 0x55ddf1ca8c00 2024-09-13T23:43:33.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f4593df3700 1 -- 172.21.15.167:0/2665389452 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf1caa160 con 0x55ddf1ca8c00 2024-09-13T23:43:33.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45aee6df40 1 -- 172.21.15.167:0/2665389452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 msgr2=0x55ddf1b51180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2665389452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 0x55ddf1b51180 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x55ddf1c51560 tx=0x55ddf196d860 comp rx=0 tx=0).stop 2024-09-13T23:43:33.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45aee6df40 1 -- 172.21.15.167:0/2665389452 shutdown_connections 2024-09-13T23:43:33.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2665389452 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 0x55ddf1b51180 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45aee6df40 1 -- 172.21.15.167:0/2665389452 >> 172.21.15.167:0/2665389452 conn(0x55ddf1ca8800 msgr2=0x55ddf196a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45aee6df40 1 -- 172.21.15.167:0/2665389452 shutdown_connections 2024-09-13T23:43:33.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45aee6df40 1 -- 172.21.15.167:0/2665389452 wait complete. 2024-09-13T23:43:33.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45aee6df40 1 Processor -- start 2024-09-13T23:43:33.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45aee6df40 1 -- start start 2024-09-13T23:43:33.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45aee6df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 0x55ddf1b51180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45aee6df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ddf18f6500 con 0x55ddf1ca8c00 2024-09-13T23:43:33.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45ace25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 0x55ddf1b51180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45ace25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 0x55ddf1b51180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52562/0 (socket says 172.21.15.167:52562) 2024-09-13T23:43:33.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45ace25700 1 -- 172.21.15.167:0/2978317345 learned_addr learned my addr 172.21.15.167:0/2978317345 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45ace25700 1 -- 172.21.15.167:0/2978317345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ddf187ad00 con 0x55ddf1ca8c00 2024-09-13T23:43:33.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.099+0000 7f45ace25700 1 --2- 172.21.15.167:0/2978317345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 0x55ddf1b51180 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55ddf1cb41e0 tx=0x55ddf1cba300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.103+0000 7f45925f0700 1 -- 172.21.15.167:0/2978317345 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf1caa2c0 con 0x55ddf1ca8c00 2024-09-13T23:43:33.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.103+0000 7f45925f0700 1 -- 172.21.15.167:0/2978317345 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ddf1c30600 con 0x55ddf1ca8c00 2024-09-13T23:43:33.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.103+0000 7f45aee6df40 1 -- 172.21.15.167:0/2978317345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ddf187a340 con 0x55ddf1ca8c00 2024-09-13T23:43:33.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.103+0000 7f45925f0700 1 -- 172.21.15.167:0/2978317345 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ddf1caa420 con 0x55ddf1ca8c00 2024-09-13T23:43:33.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.103+0000 7f45aee6df40 1 -- 172.21.15.167:0/2978317345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ddf187bd40 con 0x55ddf1ca8c00 2024-09-13T23:43:33.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.103+0000 7f45925f0700 1 -- 172.21.15.167:0/2978317345 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ddf1963b80 con 0x55ddf1ca8c00 2024-09-13T23:43:33.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.103+0000 7f45925f0700 1 --2- 172.21.15.167:0/2978317345 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1d36000 0x55ddf1d3c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.103+0000 7f45925f0700 1 -- 172.21.15.167:0/2978317345 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ddf18cf6c0 con 0x55ddf1ca8c00 2024-09-13T23:43:33.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.103+0000 7f45ade27700 1 --2- 172.21.15.167:0/2978317345 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1d36000 0x55ddf1d3c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.103+0000 7f45aee6df40 5 note: GC not initialized 2024-09-13T23:43:33.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.103+0000 7f45ade27700 1 --2- 172.21.15.167:0/2978317345 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1d36000 0x55ddf1d3c000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55ddf1c51800 tx=0x55ddf1bccd20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f458fdeb700 20 reqs_thread_entry: start 2024-09-13T23:43:33.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:33.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:33.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f458dde7700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:03 2024 2024-09-13T23:43:33.116 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:33.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f458d5c5700 20 reqs_thread_entry: start 2024-09-13T23:43:33.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-09-13T23:43:33.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1ca9800 con 0x55ddf1c9e800 2024-09-13T23:43:33.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(62 testx$tenanteduser [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 246+0+0 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x0 2024-09-13T23:43:33.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-09-13T23:43:33.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-09-13T23:43:33.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1ca9c00 con 0x55ddf1c9e800 2024-09-13T23:43:33.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-09-13T23:43:33.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-13T23:43:33.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-13T23:43:33.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-13T23:43:33.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.107+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [delete,create,call version.set in=58b,writefull 0~385 in=385b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1d36400 con 0x55ddf1c9e800 2024-09-13T23:43:33.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(64 testx$tenanteduser [delete,create,call,writefull 0~385] v33'5 uv5 ondisk = 0) v8 ==== 288+0+0 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45aee6df40 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-09-13T23:43:33.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45aee6df40 10 moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-09-13T23:43:33.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45aee6df40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-09-13T23:43:33.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:65 7.e 7:7759931f:::notify.3:head [notify cookie 94411732732160 in=588b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1d36800 con 0x55ddf1c9e000 2024-09-13T23:43:33.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== watch-notify(notify (1) cookie 94411731265664 notify 141733920785 ret 0) v3 ==== 618+0+0 (crc 0 0 0) 0x55ddf1cb6000 con 0x55ddf1c9e000 2024-09-13T23:43:33.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45a7e1b700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920785 cookie 94411731265664 notifier 4574 bl.length()=576 2024-09-13T23:43:33.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45a7e1b700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-09-13T23:43:33.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45a7e1b700 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-09-13T23:43:33.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(65 notify.3 [notify cookie 94411732732160 out=8b] v0'0 uv129 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45a7e1b700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1c9f400 con 0x55ddf1c9e000 2024-09-13T23:43:33.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== watch-notify(notify_complete (2) cookie 94411732732160 notify 141733920785 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ddf1cb61a0 con 0x55ddf1c9e000 2024-09-13T23:43:33.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(66 notify.3 [notify-ack] v0'0 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.111+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1d36c00 con 0x55ddf1c9e800 2024-09-13T23:43:33.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~22] v33'6 uv6 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45aee6df40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-13T23:43:33.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45aee6df40 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-13T23:43:33.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45aee6df40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-13T23:43:33.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94411732732160 in=204b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1d37000 con 0x55ddf1c11c00 2024-09-13T23:43:33.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94411731264512 notify 141733920790 ret 0) v3 ==== 234+0+0 (crc 0 0 0) 0x55ddf187ba00 con 0x55ddf1c11c00 2024-09-13T23:43:33.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45a761a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920790 cookie 94411731264512 notifier 4574 bl.length()=192 2024-09-13T23:43:33.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45a761a700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-13T23:43:33.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45a761a700 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-13T23:43:33.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45a761a700 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1ca9800 con 0x55ddf1c11c00 2024-09-13T23:43:33.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(68 notify.2 [notify cookie 94411732732160 out=8b] v0'0 uv129 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== watch-notify(notify_complete (2) cookie 94411732732160 notify 141733920790 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ddf1b94000 con 0x55ddf1c11c00 2024-09-13T23:43:33.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.115+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "testx$tenanteduser", 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "tenanted-user", 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "", 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 1, 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "testx$tenanteduser", 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:33.127 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:33.128 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:33.128 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:33.128 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:33.128 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:33.128 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:33.128 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:33.128 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:33.128 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:33.128 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:33.129 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:33.130 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:33.130 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:33.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.119+0000 7f45945f4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:33.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.119+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94411732729856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1d37400 con 0x55ddf1c11400 2024-09-13T23:43:33.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.119+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94411732729856] v33'127 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.119+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94411731262208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1d37800 con 0x55ddf1c11c00 2024-09-13T23:43:33.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.123+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94411731262208] v33'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.123+0000 7f45aee6df40 20 remove_watcher() i=0 2024-09-13T23:43:33.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.123+0000 7f45aee6df40 2 removed watcher, disabling cache 2024-09-13T23:43:33.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.123+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94411731263360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1d37c00 con 0x55ddf1c11400 2024-09-13T23:43:33.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.123+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94411731263360] v33'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c11400 2024-09-13T23:43:33.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.123+0000 7f45aee6df40 20 remove_watcher() i=1 2024-09-13T23:43:33.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.123+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94411731264512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1d37000 con 0x55ddf1c11c00 2024-09-13T23:43:33.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.127+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94411731264512] v33'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.127+0000 7f45aee6df40 20 remove_watcher() i=2 2024-09-13T23:43:33.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.127+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94411731265664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1d36400 con 0x55ddf1c9e000 2024-09-13T23:43:33.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.127+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94411731265664] v33'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.127+0000 7f45aee6df40 20 remove_watcher() i=3 2024-09-13T23:43:33.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.127+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94411731266816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1d36c00 con 0x55ddf1c11c00 2024-09-13T23:43:33.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.131+0000 7f45ad626700 1 -- 172.21.15.167:0/2389959210 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94411731266816] v33'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53200 con 0x55ddf1c11c00 2024-09-13T23:43:33.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.131+0000 7f45aee6df40 20 remove_watcher() i=4 2024-09-13T23:43:33.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.131+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94411731267968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1d36800 con 0x55ddf1c9e800 2024-09-13T23:43:33.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.131+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94411731267968] v33'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.131+0000 7f45aee6df40 20 remove_watcher() i=5 2024-09-13T23:43:33.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.131+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94411731269120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1ca9800 con 0x55ddf1c9e800 2024-09-13T23:43:33.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94411731269120] v33'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c53440 con 0x55ddf1c9e800 2024-09-13T23:43:33.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45aee6df40 20 remove_watcher() i=6 2024-09-13T23:43:33.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94411731270272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ddf1ca9c00 con 0x55ddf1c9e000 2024-09-13T23:43:33.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45ade27700 1 -- 172.21.15.167:0/2389959210 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94411731270272] v33'131 uv129 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ddf1c52b40 con 0x55ddf1c9e000 2024-09-13T23:43:33.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45aee6df40 20 remove_watcher() i=7 2024-09-13T23:43:33.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ddf1c11c00 msgr2=0x55ddf1b50100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ddf1c11c00 0x55ddf1b50100 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ddf1c11400 msgr2=0x55ddf1b4fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ddf1c11400 0x55ddf1b4fb80 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ddf1c9e000 msgr2=0x55ddf1b50c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ddf1c9e000 0x55ddf1b50c00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ddf1c9e800 msgr2=0x55ddf1b50680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.135+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ddf1c9e800 0x55ddf1b50680 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1c11000 msgr2=0x55ddf1b4f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1c11000 0x55ddf1b4f600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x55ddf1b49920 tx=0x55ddf196d800 comp rx=0 tx=0).stop 2024-09-13T23:43:33.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 msgr2=0x55ddf1b4eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2389959210 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869800 0x55ddf1b4eb00 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55ddf26d83c0 tx=0x55ddf1bccfc0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 reap_dead start 2024-09-13T23:43:33.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45ace25700 1 -- 172.21.15.167:0/2389959210 reap_dead start 2024-09-13T23:43:33.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 shutdown_connections 2024-09-13T23:43:33.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 >> 172.21.15.167:0/2389959210 conn(0x55ddf1869400 msgr2=0x55ddf1969800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 shutdown_connections 2024-09-13T23:43:33.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2389959210 wait complete. 2024-09-13T23:43:33.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2978317345 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1d36000 msgr2=0x55ddf1d3c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2978317345 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1d36000 0x55ddf1d3c000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x55ddf1c51800 tx=0x55ddf1bccd20 comp rx=0 tx=0).stop 2024-09-13T23:43:33.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2978317345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 msgr2=0x55ddf1b51180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2978317345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 0x55ddf1b51180 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55ddf1cb41e0 tx=0x55ddf1cba300 comp rx=0 tx=0).stop 2024-09-13T23:43:33.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2978317345 shutdown_connections 2024-09-13T23:43:33.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2978317345 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1d36000 0x55ddf1d3c000 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2978317345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1ca8c00 0x55ddf1b51180 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2978317345 >> 172.21.15.167:0/2978317345 conn(0x55ddf1ca8800 msgr2=0x55ddf196a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2978317345 shutdown_connections 2024-09-13T23:43:33.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2978317345 wait complete. 2024-09-13T23:43:33.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2559469617 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1c10000 msgr2=0x55ddf1b4f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2559469617 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1c10000 0x55ddf1b4f080 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55ddf26d8ed0 tx=0x55ddf1bccde0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2559469617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 msgr2=0x55ddf1b4e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2559469617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 0x55ddf1b4e580 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x55ddf26d80c0 tx=0x55ddf196da40 comp rx=0 tx=0).stop 2024-09-13T23:43:33.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2559469617 shutdown_connections 2024-09-13T23:43:33.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2559469617 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ddf1c10000 0x55ddf1b4f080 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 --2- 172.21.15.167:0/2559469617 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ddf1869000 0x55ddf1b4e580 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2559469617 >> 172.21.15.167:0/2559469617 conn(0x55ddf1868c00 msgr2=0x55ddf1969000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2559469617 shutdown_connections 2024-09-13T23:43:33.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.139+0000 7f45aee6df40 1 -- 172.21.15.167:0/2559469617 wait complete. 2024-09-13T23:43:33.154 DEBUG:tasks.util.rgw: json result: {'user_id': 'testx$tenanteduser', 'display_name': 'tenanted-user', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'testx$tenanteduser', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:33.154 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-09-13T23:43:33.154 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-09-13T23:43:33.154 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket /myfoo --tenant testx --uid tenanteduser 2024-09-13T23:43:33.199 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:33.199 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:33.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4baf9f5f40 1 Processor -- start 2024-09-13T23:43:33.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4baf9f5f40 1 -- start start 2024-09-13T23:43:33.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4baf9f5f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x555c16a2f400 0x555c16d14580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4baf9f5f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x555c16af3500 con 0x555c16a2f000 2024-09-13T23:43:33.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4baf9f5f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x555c16abca00 con 0x555c16a2f400 2024-09-13T23:43:33.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4bad9ad700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x555c16a2f400 0x555c16d14580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4bae1ae700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x555c16a2f000 0x555c16b2e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45528/0 (socket says 172.21.15.167:45528) 2024-09-13T23:43:33.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4bad9ad700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x555c16a2f400 0x555c16d14580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52576/0 (socket says 172.21.15.167:52576) 2024-09-13T23:43:33.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4bae1ae700 1 -- 172.21.15.167:0/1794729644 learned_addr learned my addr 172.21.15.167:0/1794729644 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:33.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4bad9ad700 1 -- 172.21.15.167:0/1794729644 >> v1:172.21.15.12:6789/0 conn(0x555c16a2f000 legacy=0x555c16b2e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4bad9ad700 1 -- 172.21.15.167:0/1794729644 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c16a41d40 con 0x555c16a2f400 2024-09-13T23:43:33.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4bad9ad700 1 --2- 172.21.15.167:0/1794729644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f400 0x555c16d14580 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x555c16d0f920 tx=0x555c16b33920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d86392c580efe5c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4bad1ac700 1 -- 172.21.15.167:0/1794729644 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ad34a0 con 0x555c16a2f400 2024-09-13T23:43:33.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4bad1ac700 1 -- 172.21.15.167:0/1794729644 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c16af3800 con 0x555c16a2f400 2024-09-13T23:43:33.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.191+0000 7f4bad1ac700 1 -- 172.21.15.167:0/1794729644 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ad3600 con 0x555c16a2f400 2024-09-13T23:43:33.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/1794729644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f400 msgr2=0x555c16d14580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/1794729644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f400 0x555c16d14580 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x555c16d0f920 tx=0x555c16b33920 comp rx=0 tx=0).stop 2024-09-13T23:43:33.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/1794729644 shutdown_connections 2024-09-13T23:43:33.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/1794729644 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f400 0x555c16d14580 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/1794729644 >> 172.21.15.167:0/1794729644 conn(0x555c16a2ec00 msgr2=0x555c16b2e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/1794729644 shutdown_connections 2024-09-13T23:43:33.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/1794729644 wait complete. 2024-09-13T23:43:33.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 Processor -- start 2024-09-13T23:43:33.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 -- start start 2024-09-13T23:43:33.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 0x555c16d14580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c16abc500 con 0x555c16a2f000 2024-09-13T23:43:33.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4bae1ae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 0x555c16d14580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4bae1ae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 0x555c16d14580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52582/0 (socket says 172.21.15.167:52582) 2024-09-13T23:43:33.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4bae1ae700 1 -- 172.21.15.167:0/2208630280 learned_addr learned my addr 172.21.15.167:0/2208630280 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4bae1ae700 1 -- 172.21.15.167:0/2208630280 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c16a41ba0 con 0x555c16a2f000 2024-09-13T23:43:33.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4bae1ae700 1 --2- 172.21.15.167:0/2208630280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 0x555c16d14580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x555c16b8be60 tx=0x555c16b33b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a281a97d5338a903 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4bac9ab700 1 -- 172.21.15.167:0/2208630280 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ad31e0 con 0x555c16a2f000 2024-09-13T23:43:33.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4bac9ab700 1 -- 172.21.15.167:0/2208630280 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c16af3b00 con 0x555c16a2f000 2024-09-13T23:43:33.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4bac9ab700 1 -- 172.21.15.167:0/2208630280 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ad34a0 con 0x555c16a2f000 2024-09-13T23:43:33.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/2208630280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 msgr2=0x555c16d14580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.195+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/2208630280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 0x555c16d14580 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x555c16b8be60 tx=0x555c16b33b60 comp rx=0 tx=0).stop 2024-09-13T23:43:33.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/2208630280 shutdown_connections 2024-09-13T23:43:33.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/2208630280 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 0x555c16d14580 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/2208630280 >> 172.21.15.167:0/2208630280 conn(0x555c16a2ec00 msgr2=0x555c16b2f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/2208630280 shutdown_connections 2024-09-13T23:43:33.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/2208630280 wait complete. 2024-09-13T23:43:33.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4baf9f5f40 1 Processor -- start 2024-09-13T23:43:33.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4baf9f5f40 1 -- start start 2024-09-13T23:43:33.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4baf9f5f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 0x555c16d14580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4baf9f5f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c16abc3c0 con 0x555c16a2f000 2024-09-13T23:43:33.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4bae1ae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 0x555c16d14580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4bae1ae700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 0x555c16d14580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52594/0 (socket says 172.21.15.167:52594) 2024-09-13T23:43:33.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4bae1ae700 1 -- 172.21.15.167:0/3807602823 learned_addr learned my addr 172.21.15.167:0/3807602823 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4bae1ae700 1 -- 172.21.15.167:0/3807602823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c16a41a00 con 0x555c16a2f000 2024-09-13T23:43:33.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4bae1ae700 1 --2- 172.21.15.167:0/3807602823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 0x555c16d14580 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x555c16b8bf50 tx=0x555c16b33a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4bab1a8700 1 -- 172.21.15.167:0/3807602823 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ad3600 con 0x555c16a2f000 2024-09-13T23:43:33.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4bab1a8700 1 -- 172.21.15.167:0/3807602823 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c16af3c80 con 0x555c16a2f000 2024-09-13T23:43:33.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/3807602823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555c16a41860 con 0x555c16a2f000 2024-09-13T23:43:33.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4bab1a8700 1 -- 172.21.15.167:0/3807602823 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ad31e0 con 0x555c16a2f000 2024-09-13T23:43:33.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.199+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/3807602823 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c16a416c0 con 0x555c16a2f000 2024-09-13T23:43:33.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 Processor -- start 2024-09-13T23:43:33.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 -- start start 2024-09-13T23:43:33.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bab1a8700 1 -- 172.21.15.167:0/3807602823 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x555c16b28b00 con 0x555c16a2f000 2024-09-13T23:43:33.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 0x555c16d14b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c16a54640 con 0x555c16a2f800 2024-09-13T23:43:33.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bae9af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 0x555c16d14b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bab1a8700 1 --2- 172.21.15.167:0/3807602823 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16de6000 0x555c16d15080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bae9af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 0x555c16d14b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52598/0 (socket says 172.21.15.167:52598) 2024-09-13T23:43:33.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bae9af700 1 -- 172.21.15.167:0/1758339201 learned_addr learned my addr 172.21.15.167:0/1758339201 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bab1a8700 1 -- 172.21.15.167:0/3807602823 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x555c16a95180 con 0x555c16a2f000 2024-09-13T23:43:33.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bae1ae700 1 --2- 172.21.15.167:0/3807602823 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16de6000 0x555c16d15080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bae9af700 1 -- 172.21.15.167:0/1758339201 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c16a40000 con 0x555c16a2f800 2024-09-13T23:43:33.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bae1ae700 1 --2- 172.21.15.167:0/3807602823 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16de6000 0x555c16d15080 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x555c16d4efc0 tx=0x555c16d98240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bae9af700 1 --2- 172.21.15.167:0/1758339201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 0x555c16d14b00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x555c16d4e600 tx=0x555c16d990e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ca8cd3ba9d9dcd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4ba89a3700 1 -- 172.21.15.167:0/1758339201 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ad3600 con 0x555c16a2f800 2024-09-13T23:43:33.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4ba89a3700 1 -- 172.21.15.167:0/1758339201 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c16dcc000 con 0x555c16a2f800 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4ba89a3700 1 -- 172.21.15.167:0/1758339201 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ad31e0 con 0x555c16a2f800 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/1758339201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 msgr2=0x555c16d14b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/1758339201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 0x555c16d14b00 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x555c16d4e600 tx=0x555c16d990e0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/1758339201 shutdown_connections 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/1758339201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 0x555c16d14b00 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/1758339201 >> 172.21.15.167:0/1758339201 conn(0x555c16a2f400 msgr2=0x555c16b2f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/1758339201 shutdown_connections 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/1758339201 wait complete. 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 Processor -- start 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 -- start start 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 0x555c16d14b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4baf9f5f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c16a548c0 con 0x555c16a2f800 2024-09-13T23:43:33.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bae9af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 0x555c16d14b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bae9af700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 0x555c16d14b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52602/0 (socket says 172.21.15.167:52602) 2024-09-13T23:43:33.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.203+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 learned_addr learned my addr 172.21.15.167:0/500248423 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c16a40ea0 con 0x555c16a2f800 2024-09-13T23:43:33.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4bae9af700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 0x555c16d14b00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x555c16d4e450 tx=0x555c16d98840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4ba71a0700 1 -- 172.21.15.167:0/500248423 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ad3a20 con 0x555c16a2f800 2024-09-13T23:43:33.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4ba71a0700 1 -- 172.21.15.167:0/500248423 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c16dcc180 con 0x555c16a2f800 2024-09-13T23:43:33.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555c16a40820 con 0x555c16a2f800 2024-09-13T23:43:33.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4ba71a0700 1 -- 172.21.15.167:0/500248423 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ad3600 con 0x555c16a2f800 2024-09-13T23:43:33.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c16a409c0 con 0x555c16a2f800 2024-09-13T23:43:33.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4ba71a0700 1 -- 172.21.15.167:0/500248423 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x555c16b28dc0 con 0x555c16a2f800 2024-09-13T23:43:33.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4ba71a0700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16de6c00 0x555c16d15600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4ba71a0700 1 -- 172.21.15.167:0/500248423 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x555c16a95340 con 0x555c16a2f800 2024-09-13T23:43:33.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4bad9ad700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16de6c00 0x555c16d15600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x555c16de7000 0x555c16d15b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7400 con 0x555c16de7000 2024-09-13T23:43:33.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.207+0000 7f4bae9af700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x555c16de7000 0x555c16d15b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4bad9ad700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16de6c00 0x555c16d15600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x555c16d0f920 tx=0x555c16b33800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4bae9af700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x555c16de7000 0x555c16d15b80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4baf9f5f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4baf9f5f40 20 realm 2024-09-13T23:43:33.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7000 2024-09-13T23:43:33.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4baf9f5f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4baf9f5f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:33.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7c00 con 0x555c16de7000 2024-09-13T23:43:33.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4baf9f5f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x555c16de7400 0x555c16d16100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7400 2024-09-13T23:43:33.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4bae1ae700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x555c16de7400 0x555c16d16100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.211+0000 7f4bae1ae700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x555c16de7400 0x555c16d16100 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4baf9f5f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:33.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7c00 con 0x555c16de7400 2024-09-13T23:43:33.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4baf9f5f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:33.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4baf9f5f40 20 searching for the correct realm 2024-09-13T23:43:33.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x555c16ea0000 0x555c16d16680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea0400 con 0x555c16ea0000 2024-09-13T23:43:33.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4bad9ad700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x555c16ea0000 0x555c16d16680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4bad9ad700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x555c16ea0000 0x555c16d16680 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea0400 con 0x555c16de7400 2024-09-13T23:43:33.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4bae1ae700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x555c16ea0400 0x555c16d16c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0400 2024-09-13T23:43:33.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.215+0000 7f4bae9af700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x555c16ea0400 0x555c16d16c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bae9af700 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x555c16ea0400 0x555c16d16c00 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7000 2024-09-13T23:43:33.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7000 2024-09-13T23:43:33.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0000 2024-09-13T23:43:33.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7000 2024-09-13T23:43:33.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7400 2024-09-13T23:43:33.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.219+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7400 2024-09-13T23:43:33.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.223+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.223+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7000 2024-09-13T23:43:33.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.223+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.223+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0400 2024-09-13T23:43:33.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.223+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.223+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7000 2024-09-13T23:43:33.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.223+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.223+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0400 2024-09-13T23:43:33.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.223+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.223+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7400 2024-09-13T23:43:33.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.223+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.223+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0000 2024-09-13T23:43:33.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.227+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.227+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7000 2024-09-13T23:43:33.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.227+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.227+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7400 2024-09-13T23:43:33.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.227+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.227+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0000 2024-09-13T23:43:33.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.227+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.227+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7000 2024-09-13T23:43:33.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.227+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.227+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7000 2024-09-13T23:43:33.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.227+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7000 2024-09-13T23:43:33.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0000 2024-09-13T23:43:33.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0400 2024-09-13T23:43:33.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7000 2024-09-13T23:43:33.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0400 2024-09-13T23:43:33.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0000 2024-09-13T23:43:33.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0400 2024-09-13T23:43:33.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.231+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7400 2024-09-13T23:43:33.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7400 2024-09-13T23:43:33.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0000 2024-09-13T23:43:33.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7400 2024-09-13T23:43:33.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16de7400 2024-09-13T23:43:33.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4baf9f5f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:33.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4baf9f5f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:33.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4baf9f5f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:33.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4baf9f5f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:33.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.235+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea0800 con 0x555c16de7000 2024-09-13T23:43:33.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea0c00 con 0x555c16de7400 2024-09-13T23:43:33.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:33.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1000 con 0x555c16de7400 2024-09-13T23:43:33.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:33.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 20 zone default found 2024-09-13T23:43:33.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 4 Realm: () 2024-09-13T23:43:33.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:33.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:33.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:33.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 20 zonegroup default 2024-09-13T23:43:33.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1400 con 0x555c16ea0400 2024-09-13T23:43:33.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1800 con 0x555c16ea0000 2024-09-13T23:43:33.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.239+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1c00 con 0x555c16ea0000 2024-09-13T23:43:33.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.243+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.243+0000 7f4baf9f5f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.243+0000 7f4baf9f5f40 20 started sync module instance, tier type = 2024-09-13T23:43:33.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.243+0000 7f4baf9f5f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:33.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.243+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1800 con 0x555c16de7400 2024-09-13T23:43:33.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.243+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.243+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93854008130304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1c00 con 0x555c16de7400 2024-09-13T23:43:33.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.243+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea0c00 con 0x555c16de7000 2024-09-13T23:43:33.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.247+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 93854008130304] v33'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.247+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.247+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93854008131456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1000 con 0x555c16de7000 2024-09-13T23:43:33.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.247+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1800 con 0x555c16de7400 2024-09-13T23:43:33.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.251+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93854008131456] v33'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.251+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.251+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93854008132608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7c00 con 0x555c16de7400 2024-09-13T23:43:33.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.251+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0400 2024-09-13T23:43:33.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.255+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.255+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93854008133760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea0800 con 0x555c16ea0400 2024-09-13T23:43:33.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.255+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1400 con 0x555c16de7400 2024-09-13T23:43:33.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.255+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 93854008132608] v33'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.255+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.255+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93854008134912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea0c00 con 0x555c16de7400 2024-09-13T23:43:33.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.255+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1000 con 0x555c16ea0000 2024-09-13T23:43:33.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.255+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 93854008133760] v33'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.259+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.259+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93854008136064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16eae000 con 0x555c16ea0000 2024-09-13T23:43:33.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.259+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16eae400 con 0x555c16ea0000 2024-09-13T23:43:33.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.259+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93854008134912] v33'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.259+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93854008136064] v33'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.259+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.259+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93854008137216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16eae800 con 0x555c16ea0000 2024-09-13T23:43:33.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.259+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16eaec00 con 0x555c16ea0400 2024-09-13T23:43:33.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93854008137216] v33'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'132 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93854008138368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16eaf000 con 0x555c16ea0400 2024-09-13T23:43:33.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4baf9f5f40 20 add_watcher() i=0 2024-09-13T23:43:33.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4baf9f5f40 20 add_watcher() i=1 2024-09-13T23:43:33.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4baf9f5f40 20 add_watcher() i=2 2024-09-13T23:43:33.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4baf9f5f40 20 add_watcher() i=3 2024-09-13T23:43:33.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4baf9f5f40 20 add_watcher() i=4 2024-09-13T23:43:33.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4baf9f5f40 20 add_watcher() i=5 2024-09-13T23:43:33.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4baf9f5f40 20 add_watcher() i=6 2024-09-13T23:43:33.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 93854008138368] v33'133 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4baf9f5f40 20 add_watcher() i=7 2024-09-13T23:43:33.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4baf9f5f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:33.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16eaf400 con 0x555c16de7000 2024-09-13T23:43:33.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.263+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93854009868288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16eaf800 con 0x555c16de7000 2024-09-13T23:43:33.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.267+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93854009868288] v33'128 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.267+0000 7f4baf9f5f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:33.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.267+0000 7f4baf9f5f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:33.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.267+0000 7f4baf9f5f40 1 Processor -- start 2024-09-13T23:43:33.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.267+0000 7f4b9517c700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:33.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.267+0000 7f4baf9f5f40 1 -- start start 2024-09-13T23:43:33.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.267+0000 7f4baf9f5f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 0x555c16d17180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.267+0000 7f4baf9f5f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c16abc3c0 con 0x555c16eaec00 2024-09-13T23:43:33.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.267+0000 7f4bad9ad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 0x555c16d17180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.267+0000 7f4bad9ad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 0x555c16d17180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52614/0 (socket says 172.21.15.167:52614) 2024-09-13T23:43:33.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.267+0000 7f4bad9ad700 1 -- 172.21.15.167:0/2325439074 learned_addr learned my addr 172.21.15.167:0/2325439074 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4bad9ad700 1 -- 172.21.15.167:0/2325439074 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c16a40b60 con 0x555c16eaec00 2024-09-13T23:43:33.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4bad9ad700 1 --2- 172.21.15.167:0/2325439074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 0x555c16d17180 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x555c16b8a120 tx=0x555c16b33860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8963720b9a5a3aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4b9497b700 1 -- 172.21.15.167:0/2325439074 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ad3760 con 0x555c16eaec00 2024-09-13T23:43:33.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4b9497b700 1 -- 172.21.15.167:0/2325439074 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c16dcc600 con 0x555c16eaec00 2024-09-13T23:43:33.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4b9497b700 1 -- 172.21.15.167:0/2325439074 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ec6000 con 0x555c16eaec00 2024-09-13T23:43:33.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/2325439074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 msgr2=0x555c16d17180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/2325439074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 0x555c16d17180 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x555c16b8a120 tx=0x555c16b33860 comp rx=0 tx=0).stop 2024-09-13T23:43:33.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/2325439074 shutdown_connections 2024-09-13T23:43:33.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/2325439074 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 0x555c16d17180 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/2325439074 >> 172.21.15.167:0/2325439074 conn(0x555c16eafc00 msgr2=0x555c16b30000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/2325439074 shutdown_connections 2024-09-13T23:43:33.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/2325439074 wait complete. 2024-09-13T23:43:33.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4baf9f5f40 1 Processor -- start 2024-09-13T23:43:33.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4baf9f5f40 1 -- start start 2024-09-13T23:43:33.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4baf9f5f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 0x555c16d17180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4baf9f5f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555c16abc500 con 0x555c16eaec00 2024-09-13T23:43:33.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4bad9ad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 0x555c16d17180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4bad9ad700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 0x555c16d17180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52622/0 (socket says 172.21.15.167:52622) 2024-09-13T23:43:33.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4bad9ad700 1 -- 172.21.15.167:0/783516576 learned_addr learned my addr 172.21.15.167:0/783516576 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.271+0000 7f4bad9ad700 1 -- 172.21.15.167:0/783516576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555c16a40d00 con 0x555c16eaec00 2024-09-13T23:43:33.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.275+0000 7f4bad9ad700 1 --2- 172.21.15.167:0/783516576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 0x555c16d17180 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x555c16d4e240 tx=0x555c16ebe300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.275+0000 7f4b93178700 1 -- 172.21.15.167:0/783516576 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ec6160 con 0x555c16eaec00 2024-09-13T23:43:33.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.275+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/783516576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555c16a40340 con 0x555c16eaec00 2024-09-13T23:43:33.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.275+0000 7f4b93178700 1 -- 172.21.15.167:0/783516576 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555c16dcc300 con 0x555c16eaec00 2024-09-13T23:43:33.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.275+0000 7f4b93178700 1 -- 172.21.15.167:0/783516576 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x555c16ec62c0 con 0x555c16eaec00 2024-09-13T23:43:33.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.275+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/783516576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555c16a41d40 con 0x555c16eaec00 2024-09-13T23:43:33.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.275+0000 7f4b93178700 1 -- 172.21.15.167:0/783516576 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x555c16b29b80 con 0x555c16eaec00 2024-09-13T23:43:33.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.275+0000 7f4b93178700 1 --2- 172.21.15.167:0/783516576 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16f38000 0x555c16f3e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.275+0000 7f4b93178700 1 -- 172.21.15.167:0/783516576 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x555c16a95500 con 0x555c16eaec00 2024-09-13T23:43:33.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.275+0000 7f4baf9f5f40 5 note: GC not initialized 2024-09-13T23:43:33.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.275+0000 7f4bae9af700 1 --2- 172.21.15.167:0/783516576 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16f38000 0x555c16f3e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.275+0000 7f4bae9af700 1 --2- 172.21.15.167:0/783516576 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16f38000 0x555c16f3e000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x555c16e49cb0 tx=0x555c16d997a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4b90973700 20 reqs_thread_entry: start 2024-09-13T23:43:33.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:33.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:33.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4b8e16e700 20 reqs_thread_entry: start 2024-09-13T23:43:33.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-09-13T23:43:33.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16eae000 con 0x555c16ea0000 2024-09-13T23:43:33.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4b8e96f700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:03 2024 2024-09-13T23:43:33.288 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:33.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(62 testx$tenanteduser [call out=48b,read 0~385 out=385b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 246+0+508 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 20 rados_obj.operate() r=0 bl.length=385 2024-09-13T23:43:33.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-09-13T23:43:33.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-09-13T23:43:33.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:33.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-13T23:43:33.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16eae400 con 0x555c16ea0400 2024-09-13T23:43:33.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv23 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:33.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-13T23:43:33.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:33.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 : miss 2024-09-13T23:43:33.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea0800 con 0x555c16ea0000 2024-09-13T23:43:33.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(64 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:33.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:33.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:33.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:33.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:33.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 2024-09-13T23:43:33.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.279+0000 7f4baf9f5f40 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : hit (requested=0x13, cached=0x13) 2024-09-13T23:43:33.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.283+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1000 con 0x555c16ea0400 2024-09-13T23:43:33.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.283+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(65 foo.buckets [call] v33'24 uv24 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.283+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:66 8.16 8:6e0d8e78:root::.bucket.meta.testx%3amyfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [create,call version.inc in=38b,writefull 0~371 in=371b,setxattr user.rgw.acl (203) in=215b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7800 con 0x555c16ea0000 2024-09-13T23:43:33.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(66 .bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [create,call,writefull 0~371,setxattr (203)] v33'1 uv1 ondisk = 0) v8 ==== 338+0+0 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4baf9f5f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x7 2024-09-13T23:43:33.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4baf9f5f40 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:33.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4baf9f5f40 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-09-13T23:43:33.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4baf9f5f40 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1, ofs0, ns] 2024-09-13T23:43:33.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.9 7:93e5b521:::notify.7:head [notify cookie 93854009870592 in=818b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16f38400 con 0x555c16ea0400 2024-09-13T23:43:33.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== watch-notify(notify (1) cookie 93854008138368 notify 141733920786 ret 0) v3 ==== 848+0+0 (crc 0 0 0) 0x555c16e4c820 con 0x555c16ea0400 2024-09-13T23:43:33.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4ba89a3700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920786 cookie 93854008138368 notifier 4582 bl.length()=806 2024-09-13T23:43:33.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4ba89a3700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x7 2024-09-13T23:43:33.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4ba89a3700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:33.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4ba89a3700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-09-13T23:43:33.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4ba89a3700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16fb0000 con 0x555c16ea0400 2024-09-13T23:43:33.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(67 notify.7 [notify cookie 93854009870592 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== watch-notify(notify_complete (2) cookie 93854009870592 notify 141733920786 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x555c16e4c9c0 con 0x555c16ea0400 2024-09-13T23:43:33.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(68 notify.7 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4baf9f5f40 20 handle_bi_update(): bucket=testx:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): orig_sources= new_sources= 2024-09-13T23:43:33.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4baf9f5f40 20 handle_bi_update(): bucket=testx:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): potential sources added=[] removed=[] 2024-09-13T23:43:33.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4baf9f5f40 20 handle_bi_update(): bucket=testx:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): orig_dests= new_dests= 2024-09-13T23:43:33.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4baf9f5f40 20 handle_bi_update(): bucket=testx:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): potential dests added=[] removed=[] 2024-09-13T23:43:33.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.287+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16f38800 con 0x555c16de7000 2024-09-13T23:43:33.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.291+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(69 testx$tenanteduser.buckets [call] v33'1 uv1 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.291+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [delete,create,call version.inc in=38b,writefull 0~193 in=193b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16f38c00 con 0x555c16de7000 2024-09-13T23:43:33.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(70 testx/myfoo [delete,create,call,writefull 0~193] v33'1 uv1 ondisk = 0) v8 ==== 281+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4baf9f5f40 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-09-13T23:43:33.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4baf9f5f40 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-09-13T23:43:33.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4baf9f5f40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-09-13T23:43:33.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify cookie 93854009870592 in=360b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16f39000 con 0x555c16de7400 2024-09-13T23:43:33.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 93854008134912 notify 141733920791 ret 0) v3 ==== 390+0+0 (crc 0 0 0) 0x555c16d96820 con 0x555c16de7400 2024-09-13T23:43:33.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4ba81a2700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920791 cookie 93854008134912 notifier 4582 bl.length()=348 2024-09-13T23:43:33.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4ba81a2700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-09-13T23:43:33.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4ba81a2700 10 rgw watcher librados: moving default.rgw.meta+root+testx/myfoo to cache LRU end 2024-09-13T23:43:33.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4ba81a2700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16de7c00 con 0x555c16de7400 2024-09-13T23:43:33.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(71 notify.4 [notify cookie 93854009870592 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(72 notify.4 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 93854009870592 notify 141733920791 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x555c16d969c0 con 0x555c16de7400 2024-09-13T23:43:33.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.295+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:73 8.9 8:91b71841:root::myfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16f39400 con 0x555c16ea0400 2024-09-13T23:43:33.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.299+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 18 ==== osd_op_reply(73 myfoo [call,delete] v33'25 uv25 ondisk = 0) v8 ==== 191+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.299+0000 7f4baf9f5f40 10 removing default.rgw.meta+root+myfoo from cache 2024-09-13T23:43:33.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.299+0000 7f4baf9f5f40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-13T23:43:33.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.299+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93854009870592 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16f39800 con 0x555c16ea0000 2024-09-13T23:43:33.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.299+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== watch-notify(notify (1) cookie 93854008137216 notify 141733920776 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x555c16f049c0 con 0x555c16ea0000 2024-09-13T23:43:33.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.299+0000 7f4ba89a3700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920776 cookie 93854008137216 notifier 4582 bl.length()=149 2024-09-13T23:43:33.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.299+0000 7f4ba89a3700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:75 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16fb0400 con 0x555c16ea0000 2024-09-13T23:43:33.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.299+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(74 notify.6 [notify cookie 93854009870592 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.299+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== watch-notify(notify_complete (2) cookie 93854009870592 notify 141733920776 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x555c16f04b60 con 0x555c16ea0000 2024-09-13T23:43:33.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.299+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(75 notify.6 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.299+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16f39c00 con 0x555c16ea0000 2024-09-13T23:43:33.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.303+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(76 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call,delete] v33'4 uv4 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.303+0000 7f4baf9f5f40 10 removing default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 from cache 2024-09-13T23:43:33.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.303+0000 7f4baf9f5f40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1, ofs0, ns] 2024-09-13T23:43:33.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.303+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:77 7.1f 7:f95f44c2:::notify.0:head [notify cookie 93854009870592 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16f39800 con 0x555c16de7400 2024-09-13T23:43:33.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.303+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== watch-notify(notify (1) cookie 93854008130304 notify 141733920792 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x555c16d969c0 con 0x555c16de7400 2024-09-13T23:43:33.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.303+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(77 notify.0 [notify cookie 93854009870592 out=8b] v0'0 uv132 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.303+0000 7f4ba81a2700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920792 cookie 93854008130304 notifier 4582 bl.length()=206 2024-09-13T23:43:33.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.303+0000 7f4ba81a2700 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16fb0000 con 0x555c16de7400 2024-09-13T23:43:33.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.303+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(78 notify.0 [notify-ack] v0'0 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.303+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== watch-notify(notify_complete (2) cookie 93854009870592 notify 141733920792 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x555c16d96b60 con 0x555c16de7400 2024-09-13T23:43:33.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.307+0000 7f4b9517c700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:33.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.307+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93854009868288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16f39000 con 0x555c16de7000 2024-09-13T23:43:33.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.307+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 93854009868288] v33'129 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.307+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93854008130304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16f38800 con 0x555c16de7400 2024-09-13T23:43:33.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.311+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 93854008130304] v33'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.311+0000 7f4baf9f5f40 20 remove_watcher() i=0 2024-09-13T23:43:33.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.311+0000 7f4baf9f5f40 2 removed watcher, disabling cache 2024-09-13T23:43:33.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.311+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93854008131456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16f38400 con 0x555c16de7000 2024-09-13T23:43:33.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.315+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 93854008131456] v33'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16de7000 2024-09-13T23:43:33.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.315+0000 7f4baf9f5f40 20 remove_watcher() i=1 2024-09-13T23:43:33.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.315+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93854008132608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16f38c00 con 0x555c16de7400 2024-09-13T23:43:33.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.315+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 93854008132608] v33'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.315+0000 7f4baf9f5f40 20 remove_watcher() i=2 2024-09-13T23:43:33.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.315+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93854008133760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea0800 con 0x555c16ea0400 2024-09-13T23:43:33.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.319+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 19 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 93854008133760] v33'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.319+0000 7f4baf9f5f40 20 remove_watcher() i=3 2024-09-13T23:43:33.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.319+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93854008134912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1c00 con 0x555c16de7400 2024-09-13T23:43:33.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.323+0000 7f4bae1ae700 1 -- 172.21.15.167:0/500248423 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 93854008134912] v33'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77200 con 0x555c16de7400 2024-09-13T23:43:33.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.323+0000 7f4baf9f5f40 20 remove_watcher() i=4 2024-09-13T23:43:33.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.323+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93854008136064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1800 con 0x555c16ea0000 2024-09-13T23:43:33.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.323+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 93854008136064] v33'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.323+0000 7f4baf9f5f40 20 remove_watcher() i=5 2024-09-13T23:43:33.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.323+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93854008137216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea1400 con 0x555c16ea0000 2024-09-13T23:43:33.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.327+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 93854008137216] v33'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e77440 con 0x555c16ea0000 2024-09-13T23:43:33.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.327+0000 7f4baf9f5f40 20 remove_watcher() i=6 2024-09-13T23:43:33.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.327+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93854008138368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x555c16ea0c00 con 0x555c16ea0400 2024-09-13T23:43:33.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4bae9af700 1 -- 172.21.15.167:0/500248423 <== osd.2 v2:172.21.15.167:6808/1180369139 20 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 93854008138368] v33'134 uv132 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x555c16e76b40 con 0x555c16ea0400 2024-09-13T23:43:33.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 20 remove_watcher() i=7 2024-09-13T23:43:33.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x555c16de7400 msgr2=0x555c16d16100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x555c16de7400 0x555c16d16100 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x555c16de7000 msgr2=0x555c16d15b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x555c16de7000 0x555c16d15b80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x555c16ea0400 msgr2=0x555c16d16c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x555c16ea0400 0x555c16d16c00 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x555c16ea0000 msgr2=0x555c16d16680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x555c16ea0000 0x555c16d16680 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16de6c00 msgr2=0x555c16d15600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16de6c00 0x555c16d15600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x555c16d0f920 tx=0x555c16b33800 comp rx=0 tx=0).stop 2024-09-13T23:43:33.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 msgr2=0x555c16d14b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/500248423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f800 0x555c16d14b00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x555c16d4e450 tx=0x555c16d98840 comp rx=0 tx=0).stop 2024-09-13T23:43:33.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 reap_dead start 2024-09-13T23:43:33.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4bad9ad700 1 -- 172.21.15.167:0/500248423 reap_dead start 2024-09-13T23:43:33.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 shutdown_connections 2024-09-13T23:43:33.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 >> 172.21.15.167:0/500248423 conn(0x555c16a2f400 msgr2=0x555c16b2f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 shutdown_connections 2024-09-13T23:43:33.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/500248423 wait complete. 2024-09-13T23:43:33.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/783516576 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16f38000 msgr2=0x555c16f3e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/783516576 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16f38000 0x555c16f3e000 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x555c16e49cb0 tx=0x555c16d997a0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/783516576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 msgr2=0x555c16d17180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/783516576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 0x555c16d17180 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x555c16d4e240 tx=0x555c16ebe300 comp rx=0 tx=0).stop 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/783516576 shutdown_connections 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/783516576 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16f38000 0x555c16f3e000 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/783516576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16eaec00 0x555c16d17180 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/783516576 >> 172.21.15.167:0/783516576 conn(0x555c16eafc00 msgr2=0x555c16b30000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/783516576 shutdown_connections 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/783516576 wait complete. 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/3807602823 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16de6000 msgr2=0x555c16d15080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/3807602823 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16de6000 0x555c16d15080 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x555c16d4efc0 tx=0x555c16d98240 comp rx=0 tx=0).stop 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/3807602823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 msgr2=0x555c16d14580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/3807602823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 0x555c16d14580 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x555c16b8bf50 tx=0x555c16b33a40 comp rx=0 tx=0).stop 2024-09-13T23:43:33.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/3807602823 shutdown_connections 2024-09-13T23:43:33.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/3807602823 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555c16de6000 0x555c16d15080 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 --2- 172.21.15.167:0/3807602823 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555c16a2f000 0x555c16d14580 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/3807602823 >> 172.21.15.167:0/3807602823 conn(0x555c16a2ec00 msgr2=0x555c16b2f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/3807602823 shutdown_connections 2024-09-13T23:43:33.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.331+0000 7f4baf9f5f40 1 -- 172.21.15.167:0/3807602823 wait complete. 2024-09-13T23:43:33.346 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:testx/myfoo'] 2024-09-13T23:43:33.347 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:testx/myfoo'] 2024-09-13T23:43:33.347 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:testx/myfoo 2024-09-13T23:43:33.387 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:33.387 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:33.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f4151e6af40 1 Processor -- start 2024-09-13T23:43:33.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f4151e6af40 1 -- start start 2024-09-13T23:43:33.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f4151e6af40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c2c2111000 0x55c2c23f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f4151e6af40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c2c21d5500 con 0x55c2c2111400 2024-09-13T23:43:33.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f4151e6af40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c2c219ea00 con 0x55c2c2111000 2024-09-13T23:43:33.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f4150623700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c2c2111000 0x55c2c23f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f4150623700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c2c2111000 0x55c2c23f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52638/0 (socket says 172.21.15.167:52638) 2024-09-13T23:43:33.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f414fe22700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55c2c2111400 0x55c2c2210800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45532/0 (socket says 172.21.15.167:45532) 2024-09-13T23:43:33.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f4150623700 1 -- 172.21.15.167:0/1601766737 learned_addr learned my addr 172.21.15.167:0/1601766737 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f4150623700 1 -- 172.21.15.167:0/1601766737 >> v1:172.21.15.12:6789/0 conn(0x55c2c2111400 legacy=0x55c2c2210800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f4150623700 1 -- 172.21.15.167:0/1601766737 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c2123d40 con 0x55c2c2111000 2024-09-13T23:43:33.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f4150623700 1 --2- 172.21.15.167:0/1601766737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111000 0x55c2c23f6580 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55c2c23f1a70 tx=0x55c2c2215920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06833505065948 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f414f621700 1 -- 172.21.15.167:0/1601766737 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c21b54a0 con 0x55c2c2111000 2024-09-13T23:43:33.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f414f621700 1 -- 172.21.15.167:0/1601766737 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c2c21d5800 con 0x55c2c2111000 2024-09-13T23:43:33.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.379+0000 7f414f621700 1 -- 172.21.15.167:0/1601766737 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c21b5600 con 0x55c2c2111000 2024-09-13T23:43:33.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4151e6af40 1 -- 172.21.15.167:0/1601766737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111000 msgr2=0x55c2c23f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4151e6af40 1 --2- 172.21.15.167:0/1601766737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111000 0x55c2c23f6580 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55c2c23f1a70 tx=0x55c2c2215920 comp rx=0 tx=0).stop 2024-09-13T23:43:33.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4151e6af40 1 -- 172.21.15.167:0/1601766737 shutdown_connections 2024-09-13T23:43:33.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4151e6af40 1 --2- 172.21.15.167:0/1601766737 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111000 0x55c2c23f6580 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4151e6af40 1 -- 172.21.15.167:0/1601766737 >> 172.21.15.167:0/1601766737 conn(0x55c2c2110c00 msgr2=0x55c2c2210000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4151e6af40 1 -- 172.21.15.167:0/1601766737 shutdown_connections 2024-09-13T23:43:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4151e6af40 1 -- 172.21.15.167:0/1601766737 wait complete. 2024-09-13T23:43:33.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4151e6af40 1 Processor -- start 2024-09-13T23:43:33.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4151e6af40 1 -- start start 2024-09-13T23:43:33.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4151e6af40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111000 0x55c2c23f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4151e6af40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c219e500 con 0x55c2c2111000 2024-09-13T23:43:33.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4150623700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111000 0x55c2c23f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4150623700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111000 0x55c2c23f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52646/0 (socket says 172.21.15.167:52646) 2024-09-13T23:43:33.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4150623700 1 -- 172.21.15.167:0/544582940 learned_addr learned my addr 172.21.15.167:0/544582940 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4150623700 1 -- 172.21.15.167:0/544582940 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c2123ba0 con 0x55c2c2111000 2024-09-13T23:43:33.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f4150623700 1 --2- 172.21.15.167:0/544582940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111000 0x55c2c23f6580 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55c2c23f0c30 tx=0x55c2c2215800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987d14a195b90324 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f414ee20700 1 -- 172.21.15.167:0/544582940 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c21b5760 con 0x55c2c2111000 2024-09-13T23:43:33.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f414ee20700 1 -- 172.21.15.167:0/544582940 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c2c21d5b00 con 0x55c2c2111000 2024-09-13T23:43:33.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.383+0000 7f414ee20700 1 -- 172.21.15.167:0/544582940 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c21b54a0 con 0x55c2c2111000 2024-09-13T23:43:33.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 -- 172.21.15.167:0/544582940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111000 msgr2=0x55c2c23f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 --2- 172.21.15.167:0/544582940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111000 0x55c2c23f6580 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x55c2c23f0c30 tx=0x55c2c2215800 comp rx=0 tx=0).stop 2024-09-13T23:43:33.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 -- 172.21.15.167:0/544582940 shutdown_connections 2024-09-13T23:43:33.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 --2- 172.21.15.167:0/544582940 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111000 0x55c2c23f6580 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 -- 172.21.15.167:0/544582940 >> 172.21.15.167:0/544582940 conn(0x55c2c2110c00 msgr2=0x55c2c2211000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 -- 172.21.15.167:0/544582940 shutdown_connections 2024-09-13T23:43:33.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 -- 172.21.15.167:0/544582940 wait complete. 2024-09-13T23:43:33.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 Processor -- start 2024-09-13T23:43:33.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 -- start start 2024-09-13T23:43:33.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111400 0x55c2c23f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c219e3c0 con 0x55c2c2111400 2024-09-13T23:43:33.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4150623700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111400 0x55c2c23f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4150623700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111400 0x55c2c23f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52662/0 (socket says 172.21.15.167:52662) 2024-09-13T23:43:33.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4150623700 1 -- 172.21.15.167:0/902077636 learned_addr learned my addr 172.21.15.167:0/902077636 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4150623700 1 -- 172.21.15.167:0/902077636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c2123a00 con 0x55c2c2111400 2024-09-13T23:43:33.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4150623700 1 --2- 172.21.15.167:0/902077636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111400 0x55c2c23f6b00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55c2c226c0f0 tx=0x55c2c2215920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f414d61d700 1 -- 172.21.15.167:0/902077636 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c21b5600 con 0x55c2c2111400 2024-09-13T23:43:33.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f414d61d700 1 -- 172.21.15.167:0/902077636 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c2c21d5c80 con 0x55c2c2111400 2024-09-13T23:43:33.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 -- 172.21.15.167:0/902077636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c2c2123860 con 0x55c2c2111400 2024-09-13T23:43:33.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f414d61d700 1 -- 172.21.15.167:0/902077636 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c21b58c0 con 0x55c2c2111400 2024-09-13T23:43:33.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.387+0000 7f4151e6af40 1 -- 172.21.15.167:0/902077636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2c21236c0 con 0x55c2c2111400 2024-09-13T23:43:33.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4151e6af40 1 Processor -- start 2024-09-13T23:43:33.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f414d61d700 1 -- 172.21.15.167:0/902077636 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c2c220ab00 con 0x55c2c2111400 2024-09-13T23:43:33.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4151e6af40 1 -- start start 2024-09-13T23:43:33.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f414d61d700 1 --2- 172.21.15.167:0/902077636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c2111c00 0x55c2c23f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f414d61d700 1 -- 172.21.15.167:0/902077636 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c2c2177180 con 0x55c2c2111400 2024-09-13T23:43:33.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4151e6af40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 0x55c2c23f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4151e6af40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c219e000 con 0x55c2c24ae000 2024-09-13T23:43:33.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4150e24700 1 --2- 172.21.15.167:0/902077636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c2111c00 0x55c2c23f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4150623700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 0x55c2c23f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4150623700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 0x55c2c23f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52668/0 (socket says 172.21.15.167:52668) 2024-09-13T23:43:33.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4150623700 1 -- 172.21.15.167:0/2183319344 learned_addr learned my addr 172.21.15.167:0/2183319344 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4150e24700 1 --2- 172.21.15.167:0/902077636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c2111c00 0x55c2c23f7080 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55c2c24387b0 tx=0x55c2c2474b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4150623700 1 -- 172.21.15.167:0/2183319344 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c2122000 con 0x55c2c24ae000 2024-09-13T23:43:33.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4150623700 1 --2- 172.21.15.167:0/2183319344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 0x55c2c23f7600 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x55c2c2438270 tx=0x55c2c2474c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642f07258641f059 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f414ae18700 1 -- 172.21.15.167:0/2183319344 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c21b5600 con 0x55c2c24ae000 2024-09-13T23:43:33.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f414ae18700 1 -- 172.21.15.167:0/2183319344 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c2c21d5c80 con 0x55c2c24ae000 2024-09-13T23:43:33.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f414ae18700 1 -- 172.21.15.167:0/2183319344 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c21b58c0 con 0x55c2c24ae000 2024-09-13T23:43:33.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4151e6af40 1 -- 172.21.15.167:0/2183319344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 msgr2=0x55c2c23f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4151e6af40 1 --2- 172.21.15.167:0/2183319344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 0x55c2c23f7600 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x55c2c2438270 tx=0x55c2c2474c60 comp rx=0 tx=0).stop 2024-09-13T23:43:33.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4151e6af40 1 -- 172.21.15.167:0/2183319344 shutdown_connections 2024-09-13T23:43:33.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4151e6af40 1 --2- 172.21.15.167:0/2183319344 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 0x55c2c23f7600 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.391+0000 7f4151e6af40 1 -- 172.21.15.167:0/2183319344 >> 172.21.15.167:0/2183319344 conn(0x55c2c2111800 msgr2=0x55c2c2211800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4151e6af40 1 -- 172.21.15.167:0/2183319344 shutdown_connections 2024-09-13T23:43:33.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4151e6af40 1 -- 172.21.15.167:0/2183319344 wait complete. 2024-09-13T23:43:33.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4151e6af40 1 Processor -- start 2024-09-13T23:43:33.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4151e6af40 1 -- start start 2024-09-13T23:43:33.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4151e6af40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 0x55c2c23f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4151e6af40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c2136640 con 0x55c2c24ae000 2024-09-13T23:43:33.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4150623700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 0x55c2c23f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4150623700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 0x55c2c23f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52676/0 (socket says 172.21.15.167:52676) 2024-09-13T23:43:33.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 learned_addr learned my addr 172.21.15.167:0/2337603237 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c2122ea0 con 0x55c2c24ae000 2024-09-13T23:43:33.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4150623700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 0x55c2c23f7600 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55c2c24384e0 tx=0x55c2c2474e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4149615700 1 -- 172.21.15.167:0/2337603237 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c21b5760 con 0x55c2c24ae000 2024-09-13T23:43:33.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4149615700 1 -- 172.21.15.167:0/2337603237 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c2c24d4000 con 0x55c2c24ae000 2024-09-13T23:43:33.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4149615700 1 -- 172.21.15.167:0/2337603237 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c21b54a0 con 0x55c2c24ae000 2024-09-13T23:43:33.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c2c2122820 con 0x55c2c24ae000 2024-09-13T23:43:33.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.395+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2c21229c0 con 0x55c2c24ae000 2024-09-13T23:43:33.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4149615700 1 -- 172.21.15.167:0/2337603237 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c2c220adc0 con 0x55c2c24ae000 2024-09-13T23:43:33.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4149615700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c24af800 0x55c2c23f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4149615700 1 -- 172.21.15.167:0/2337603237 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c2c2177340 con 0x55c2c24ae000 2024-09-13T23:43:33.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f414fe22700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c24af800 0x55c2c23f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4151e6af40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4151e6af40 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c2c24ae400 0x55c2c23f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c24ae800 con 0x55c2c24ae400 2024-09-13T23:43:33.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f414fe22700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c24af800 0x55c2c23f8680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55c2c23f1f80 tx=0x55c2c2536000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4150e24700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c2c24ae400 0x55c2c23f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4150e24700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c2c24ae400 0x55c2c23f7b80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4151e6af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4151e6af40 20 realm 2024-09-13T23:43:33.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4151e6af40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.399+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c24aec00 con 0x55c2c24ae400 2024-09-13T23:43:33.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:33.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c24ae800 con 0x55c2c24ae400 2024-09-13T23:43:33.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c2c24aec00 0x55c2c23f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c24afc00 con 0x55c2c24aec00 2024-09-13T23:43:33.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4150623700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c2c24aec00 0x55c2c23f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4150623700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c2c24aec00 0x55c2c23f8100 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:33.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550000 con 0x55c2c24aec00 2024-09-13T23:43:33.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:33.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 20 searching for the correct realm 2024-09-13T23:43:33.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c2c2550400 0x55c2c23f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550400 2024-09-13T23:43:33.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.403+0000 7f414fe22700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c2c2550400 0x55c2c23f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f414fe22700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c2c2550400 0x55c2c23f8c00 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24aec00 2024-09-13T23:43:33.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f4150623700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c2c2550000 0x55c2c23f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550000 2024-09-13T23:43:33.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f4150e24700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c2c2550000 0x55c2c23f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f4150e24700 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c2c2550000 0x55c2c23f6580 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24ae400 2024-09-13T23:43:33.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24ae400 2024-09-13T23:43:33.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.407+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550400 2024-09-13T23:43:33.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24ae400 2024-09-13T23:43:33.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24aec00 2024-09-13T23:43:33.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24aec00 2024-09-13T23:43:33.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24ae400 2024-09-13T23:43:33.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550000 2024-09-13T23:43:33.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24ae400 2024-09-13T23:43:33.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.411+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550000 2024-09-13T23:43:33.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.415+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.415+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24aec00 2024-09-13T23:43:33.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.415+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.415+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550400 2024-09-13T23:43:33.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.415+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.415+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24ae400 2024-09-13T23:43:33.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.415+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.415+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24aec00 2024-09-13T23:43:33.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.415+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.415+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550400 2024-09-13T23:43:33.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.415+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.415+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24ae400 2024-09-13T23:43:33.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24ae400 2024-09-13T23:43:33.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24ae400 2024-09-13T23:43:33.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550400 2024-09-13T23:43:33.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550000 2024-09-13T23:43:33.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24ae400 2024-09-13T23:43:33.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550000 2024-09-13T23:43:33.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550400 2024-09-13T23:43:33.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.419+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550000 2024-09-13T23:43:33.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.423+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.423+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24aec00 2024-09-13T23:43:33.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.423+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.423+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24aec00 2024-09-13T23:43:33.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.423+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.423+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c2550400 2024-09-13T23:43:33.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.423+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.423+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24aec00 2024-09-13T23:43:33.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.423+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.423+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24aec00 2024-09-13T23:43:33.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:33.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:33.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:33.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:33.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550c00 con 0x55c2c24ae400 2024-09-13T23:43:33.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551000 con 0x55c2c24aec00 2024-09-13T23:43:33.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:33.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551400 con 0x55c2c24aec00 2024-09-13T23:43:33.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:33.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 zone default found 2024-09-13T23:43:33.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 4 Realm: () 2024-09-13T23:43:33.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:33.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:33.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:33.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 zonegroup default 2024-09-13T23:43:33.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551800 con 0x55c2c2550000 2024-09-13T23:43:33.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.427+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551c00 con 0x55c2c2550400 2024-09-13T23:43:33.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.431+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.431+0000 7f4151e6af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.431+0000 7f4151e6af40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.431+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550c00 con 0x55c2c2550400 2024-09-13T23:43:33.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.431+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.431+0000 7f4151e6af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.431+0000 7f4151e6af40 20 started sync module instance, tier type = 2024-09-13T23:43:33.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.431+0000 7f4151e6af40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:33.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.431+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551800 con 0x55c2c24aec00 2024-09-13T23:43:33.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.435+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.435+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94294970915584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c24ae800 con 0x55c2c24aec00 2024-09-13T23:43:33.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.435+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24ae400 2024-09-13T23:43:33.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.435+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94294970915584] v33'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.435+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.435+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94294970916736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551000 con 0x55c2c24ae400 2024-09-13T23:43:33.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.435+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551400 con 0x55c2c24aec00 2024-09-13T23:43:33.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.439+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94294970916736] v33'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.439+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.439+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94294970917888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551800 con 0x55c2c24aec00 2024-09-13T23:43:33.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.439+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551c00 con 0x55c2c2550000 2024-09-13T23:43:33.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.443+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.443+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94294970919040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550c00 con 0x55c2c2550000 2024-09-13T23:43:33.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.443+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c24afc00 con 0x55c2c24aec00 2024-09-13T23:43:33.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.443+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94294970917888] v33'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.443+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94294970919040] v33'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.443+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.443+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94294970920192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550800 con 0x55c2c24aec00 2024-09-13T23:43:33.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.443+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551000 con 0x55c2c2550400 2024-09-13T23:43:33.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.447+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.447+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94294970921344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551c00 con 0x55c2c2550400 2024-09-13T23:43:33.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.447+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550c00 con 0x55c2c2550400 2024-09-13T23:43:33.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.447+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94294970920192] v33'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.447+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94294970921344] v33'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.447+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.447+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94294970922496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551000 con 0x55c2c2550400 2024-09-13T23:43:33.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.447+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2551c00 con 0x55c2c2550000 2024-09-13T23:43:33.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'135 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94294970923648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c2550c00 con 0x55c2c2550000 2024-09-13T23:43:33.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4151e6af40 20 add_watcher() i=0 2024-09-13T23:43:33.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4151e6af40 20 add_watcher() i=1 2024-09-13T23:43:33.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4151e6af40 20 add_watcher() i=2 2024-09-13T23:43:33.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4151e6af40 20 add_watcher() i=3 2024-09-13T23:43:33.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4151e6af40 20 add_watcher() i=4 2024-09-13T23:43:33.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4151e6af40 20 add_watcher() i=5 2024-09-13T23:43:33.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94294970922496] v33'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4151e6af40 20 add_watcher() i=6 2024-09-13T23:43:33.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94294970923648] v33'136 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4151e6af40 20 add_watcher() i=7 2024-09-13T23:43:33.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4151e6af40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:33.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.451+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c255c000 con 0x55c2c24ae400 2024-09-13T23:43:33.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94294972432384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c255c400 con 0x55c2c24ae400 2024-09-13T23:43:33.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94294972432384] v33'130 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f41375f1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:33.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f4151e6af40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:33.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f4151e6af40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:33.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f4151e6af40 1 Processor -- start 2024-09-13T23:43:33.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f4151e6af40 1 -- start start 2024-09-13T23:43:33.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f4151e6af40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 0x55c2c23f9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f4151e6af40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c2136780 con 0x55c2c255cc00 2024-09-13T23:43:33.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f414fe22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 0x55c2c23f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f414fe22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 0x55c2c23f9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52692/0 (socket says 172.21.15.167:52692) 2024-09-13T23:43:33.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.455+0000 7f414fe22700 1 -- 172.21.15.167:0/1125078899 learned_addr learned my addr 172.21.15.167:0/1125078899 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f414fe22700 1 -- 172.21.15.167:0/1125078899 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c2122b60 con 0x55c2c255cc00 2024-09-13T23:43:33.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f414fe22700 1 --2- 172.21.15.167:0/1125078899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 0x55c2c23f9180 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55c2c2568060 tx=0x55c2c2475f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97378d5f239df66 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4136df0700 1 -- 172.21.15.167:0/1125078899 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c255e160 con 0x55c2c255cc00 2024-09-13T23:43:33.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4136df0700 1 -- 172.21.15.167:0/1125078899 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c2c24d4a80 con 0x55c2c255cc00 2024-09-13T23:43:33.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4136df0700 1 -- 172.21.15.167:0/1125078899 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c255e2c0 con 0x55c2c255cc00 2024-09-13T23:43:33.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4151e6af40 1 -- 172.21.15.167:0/1125078899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 msgr2=0x55c2c23f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4151e6af40 1 --2- 172.21.15.167:0/1125078899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 0x55c2c23f9180 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x55c2c2568060 tx=0x55c2c2475f20 comp rx=0 tx=0).stop 2024-09-13T23:43:33.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4151e6af40 1 -- 172.21.15.167:0/1125078899 shutdown_connections 2024-09-13T23:43:33.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4151e6af40 1 --2- 172.21.15.167:0/1125078899 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 0x55c2c23f9180 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4151e6af40 1 -- 172.21.15.167:0/1125078899 >> 172.21.15.167:0/1125078899 conn(0x55c2c255c800 msgr2=0x55c2c2212000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4151e6af40 1 -- 172.21.15.167:0/1125078899 shutdown_connections 2024-09-13T23:43:33.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4151e6af40 1 -- 172.21.15.167:0/1125078899 wait complete. 2024-09-13T23:43:33.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4151e6af40 1 Processor -- start 2024-09-13T23:43:33.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4151e6af40 1 -- start start 2024-09-13T23:43:33.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4151e6af40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 0x55c2c23f9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f4151e6af40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c2c219e000 con 0x55c2c255cc00 2024-09-13T23:43:33.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f414fe22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 0x55c2c23f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f414fe22700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 0x55c2c23f9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52708/0 (socket says 172.21.15.167:52708) 2024-09-13T23:43:33.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.459+0000 7f414fe22700 1 -- 172.21.15.167:0/896950295 learned_addr learned my addr 172.21.15.167:0/896950295 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f414fe22700 1 -- 172.21.15.167:0/896950295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c2c2122d00 con 0x55c2c255cc00 2024-09-13T23:43:33.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f414fe22700 1 --2- 172.21.15.167:0/896950295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 0x55c2c23f9180 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55c2c2569320 tx=0x55c2c25360c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f41355ed700 1 -- 172.21.15.167:0/896950295 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c255e420 con 0x55c2c255cc00 2024-09-13T23:43:33.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f41355ed700 1 -- 172.21.15.167:0/896950295 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c2c24d4780 con 0x55c2c255cc00 2024-09-13T23:43:33.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f41355ed700 1 -- 172.21.15.167:0/896950295 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c2c255e160 con 0x55c2c255cc00 2024-09-13T23:43:33.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f4151e6af40 1 -- 172.21.15.167:0/896950295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c2c2122340 con 0x55c2c255cc00 2024-09-13T23:43:33.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f4151e6af40 1 -- 172.21.15.167:0/896950295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c2c2122b60 con 0x55c2c255cc00 2024-09-13T23:43:33.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f41355ed700 1 -- 172.21.15.167:0/896950295 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c2c220b8c0 con 0x55c2c255cc00 2024-09-13T23:43:33.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f41355ed700 1 --2- 172.21.15.167:0/896950295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c25ec000 0x55c2c25f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f41355ed700 1 -- 172.21.15.167:0/896950295 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55c2c21776c0 con 0x55c2c255cc00 2024-09-13T23:43:33.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f4151e6af40 5 note: GC not initialized 2024-09-13T23:43:33.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f4150e24700 1 --2- 172.21.15.167:0/896950295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c25ec000 0x55c2c25f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.463+0000 7f4150e24700 1 --2- 172.21.15.167:0/896950295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c25ec000 0x55c2c25f2000 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55c2c24eb410 tx=0x55c2c2474a80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.467+0000 7f4132de8700 20 reqs_thread_entry: start 2024-09-13T23:43:33.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.467+0000 7f4151e6af40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:33.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.467+0000 7f4151e6af40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:33.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.467+0000 7f4130de4700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:03 2024 2024-09-13T23:43:33.476 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:33.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.467+0000 7f41305c2700 20 reqs_thread_entry: start 2024-09-13T23:43:33.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.467+0000 7f4151e6af40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.467+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c255d800 con 0x55c2c24ae400 2024-09-13T23:43:33.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.467+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 testx/myfoo [call out=48b,stat out=16b,read 0~193 out=193b,getxattrs out=75b] v0'0 uv1 ondisk = 0) v8 ==== 281+0+332 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.467+0000 7f4151e6af40 20 rados_obj.operate() r=0 bl.length=193 2024-09-13T23:43:33.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.467+0000 7f41375f1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:33.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.467+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94294972432384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c255dc00 con 0x55c2c24ae400 2024-09-13T23:43:33.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.471+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94294972432384] v33'131 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.471+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94294970915584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c25ec400 con 0x55c2c24aec00 2024-09-13T23:43:33.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.475+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94294970915584] v33'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.475+0000 7f4151e6af40 20 remove_watcher() i=0 2024-09-13T23:43:33.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.475+0000 7f4151e6af40 2 removed watcher, disabling cache 2024-09-13T23:43:33.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.475+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94294970916736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c25ec800 con 0x55c2c24ae400 2024-09-13T23:43:33.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.475+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94294970916736] v33'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c24ae400 2024-09-13T23:43:33.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.475+0000 7f4151e6af40 20 remove_watcher() i=1 2024-09-13T23:43:33.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.475+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94294970917888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c25ecc00 con 0x55c2c24aec00 2024-09-13T23:43:33.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.479+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94294970917888] v33'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.479+0000 7f4151e6af40 20 remove_watcher() i=2 2024-09-13T23:43:33.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.479+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94294970919040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c25ed000 con 0x55c2c2550000 2024-09-13T23:43:33.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.479+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94294970919040] v33'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.479+0000 7f4151e6af40 20 remove_watcher() i=3 2024-09-13T23:43:33.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.479+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94294970920192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c25ed400 con 0x55c2c24aec00 2024-09-13T23:43:33.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.483+0000 7f4150623700 1 -- 172.21.15.167:0/2337603237 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94294970920192] v33'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549200 con 0x55c2c24aec00 2024-09-13T23:43:33.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.483+0000 7f4151e6af40 20 remove_watcher() i=4 2024-09-13T23:43:33.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.483+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94294970921344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c25ed800 con 0x55c2c2550400 2024-09-13T23:43:33.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.487+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94294970921344] v33'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.487+0000 7f4151e6af40 20 remove_watcher() i=5 2024-09-13T23:43:33.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.487+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94294970922496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c25edc00 con 0x55c2c2550400 2024-09-13T23:43:33.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.487+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94294970922496] v33'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2549440 con 0x55c2c2550400 2024-09-13T23:43:33.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.487+0000 7f4151e6af40 20 remove_watcher() i=6 2024-09-13T23:43:33.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.487+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94294970923648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55c2c255c000 con 0x55c2c2550000 2024-09-13T23:43:33.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4150e24700 1 -- 172.21.15.167:0/2337603237 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94294970923648] v33'137 uv135 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c2c2548b40 con 0x55c2c2550000 2024-09-13T23:43:33.500 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 20 remove_watcher() i=7 2024-09-13T23:43:33.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c2c24aec00 msgr2=0x55c2c23f8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c2c24aec00 0x55c2c23f8100 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c2c24ae400 msgr2=0x55c2c23f7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c2c24ae400 0x55c2c23f7b80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c2c2550000 msgr2=0x55c2c23f6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c2c2550000 0x55c2c23f6580 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c2c2550400 msgr2=0x55c2c23f8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c2c2550400 0x55c2c23f8c00 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c24af800 msgr2=0x55c2c23f8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c24af800 0x55c2c23f8680 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55c2c23f1f80 tx=0x55c2c2536000 comp rx=0 tx=0).stop 2024-09-13T23:43:33.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 msgr2=0x55c2c23f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/2337603237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c24ae000 0x55c2c23f7600 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x55c2c24384e0 tx=0x55c2c2474e40 comp rx=0 tx=0).stop 2024-09-13T23:43:33.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 reap_dead start 2024-09-13T23:43:33.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 shutdown_connections 2024-09-13T23:43:33.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 >> 172.21.15.167:0/2337603237 conn(0x55c2c2111800 msgr2=0x55c2c2211800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f414fe22700 1 -- 172.21.15.167:0/2337603237 reap_dead start 2024-09-13T23:43:33.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 shutdown_connections 2024-09-13T23:43:33.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/2337603237 wait complete. 2024-09-13T23:43:33.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/896950295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c25ec000 msgr2=0x55c2c25f2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/896950295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c25ec000 0x55c2c25f2000 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x55c2c24eb410 tx=0x55c2c2474a80 comp rx=0 tx=0).stop 2024-09-13T23:43:33.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/896950295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 msgr2=0x55c2c23f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/896950295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 0x55c2c23f9180 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x55c2c2569320 tx=0x55c2c25360c0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/896950295 shutdown_connections 2024-09-13T23:43:33.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/896950295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c25ec000 0x55c2c25f2000 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/896950295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c255cc00 0x55c2c23f9180 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/896950295 >> 172.21.15.167:0/896950295 conn(0x55c2c255c800 msgr2=0x55c2c2212000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/896950295 shutdown_connections 2024-09-13T23:43:33.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/896950295 wait complete. 2024-09-13T23:43:33.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/902077636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c2111c00 msgr2=0x55c2c23f7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/902077636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c2111c00 0x55c2c23f7080 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55c2c24387b0 tx=0x55c2c2474b40 comp rx=0 tx=0).stop 2024-09-13T23:43:33.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/902077636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111400 msgr2=0x55c2c23f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/902077636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111400 0x55c2c23f6b00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55c2c226c0f0 tx=0x55c2c2215920 comp rx=0 tx=0).stop 2024-09-13T23:43:33.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/902077636 shutdown_connections 2024-09-13T23:43:33.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/902077636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c2c2111c00 0x55c2c23f7080 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 --2- 172.21.15.167:0/902077636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c2c2111400 0x55c2c23f6b00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.491+0000 7f4151e6af40 1 -- 172.21.15.167:0/902077636 >> 172.21.15.167:0/902077636 conn(0x55c2c2110c00 msgr2=0x55c2c2211000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.495+0000 7f4151e6af40 1 -- 172.21.15.167:0/902077636 shutdown_connections 2024-09-13T23:43:33.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.495+0000 7f4151e6af40 1 -- 172.21.15.167:0/902077636 wait complete. 2024-09-13T23:43:33.505 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:33.506 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "bucket:testx/myfoo", 2024-09-13T23:43:33.506 INFO:teuthology.orchestra.run.smithi167.stdout: "ver": { 2024-09-13T23:43:33.506 INFO:teuthology.orchestra.run.smithi167.stdout: "tag": "sqh9RANAUiiP320OhWWsUodW", 2024-09-13T23:43:33.506 INFO:teuthology.orchestra.run.smithi167.stdout: "ver": 2 2024-09-13T23:43:33.506 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:33.506 INFO:teuthology.orchestra.run.smithi167.stdout: "mtime": "2024-09-13T23:43:33.296265Z", 2024-09-13T23:43:33.506 INFO:teuthology.orchestra.run.smithi167.stdout: "data": { 2024-09-13T23:43:33.506 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": { 2024-09-13T23:43:33.506 INFO:teuthology.orchestra.run.smithi167.stdout: "name": "myfoo", 2024-09-13T23:43:33.506 INFO:teuthology.orchestra.run.smithi167.stdout: "marker": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1", 2024-09-13T23:43:33.506 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_id": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1", 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout: "tenant": "testx", 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout: "explicit_placement": { 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": "", 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "", 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": "" 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "testx$tenanteduser", 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout: "creation_time": "2024-09-13T23:43:29.849424Z", 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout: "linked": true, 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout: "has_bucket_info": false 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:33.507 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:33.508 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:testx/myfoo', 'ver': {'tag': 'sqh9RANAUiiP320OhWWsUodW', 'ver': 2}, 'mtime': '2024-09-13T23:43:33.296265Z', 'data': {'bucket': {'name': 'myfoo', 'marker': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1', 'bucket_id': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1', 'tenant': 'testx', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'testx$tenanteduser', 'creation_time': '2024-09-13T23:43:29.849424Z', 'linked': True, 'has_bucket_info': False}} 2024-09-13T23:43:33.509 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-09-13T23:43:33.509 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-09-13T23:43:33.509 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket testx/myfoo --uid foo 2024-09-13T23:43:33.594 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:33.594 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:33.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d4248cf40 1 Processor -- start 2024-09-13T23:43:33.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d4248cf40 1 -- start start 2024-09-13T23:43:33.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d4248cf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55e3fa437400 0x55e3fa71c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d4248cf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e3fa4fb500 con 0x55e3fa437000 2024-09-13T23:43:33.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d4248cf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e3fa4c4a00 con 0x55e3fa437400 2024-09-13T23:43:33.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d40444700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55e3fa437400 0x55e3fa71c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d40444700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55e3fa437400 0x55e3fa71c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52714/0 (socket says 172.21.15.167:52714) 2024-09-13T23:43:33.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d40c45700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55e3fa437000 0x55e3fa536800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45534/0 (socket says 172.21.15.167:45534) 2024-09-13T23:43:33.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d40444700 1 -- 172.21.15.167:0/123587811 learned_addr learned my addr 172.21.15.167:0/123587811 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d40444700 1 -- 172.21.15.167:0/123587811 >> v1:172.21.15.12:6789/0 conn(0x55e3fa437000 legacy=0x55e3fa536800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d40444700 1 -- 172.21.15.167:0/123587811 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3fa449d40 con 0x55e3fa437400 2024-09-13T23:43:33.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d40444700 1 --2- 172.21.15.167:0/123587811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437400 0x55e3fa71c580 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55e3fa717920 tx=0x55e3fa53b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb19282a94fcf24 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d3fc43700 1 -- 172.21.15.167:0/123587811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa4db4a0 con 0x55e3fa437400 2024-09-13T23:43:33.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d3fc43700 1 -- 172.21.15.167:0/123587811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e3fa4fb800 con 0x55e3fa437400 2024-09-13T23:43:33.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d3fc43700 1 -- 172.21.15.167:0/123587811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa4db600 con 0x55e3fa437400 2024-09-13T23:43:33.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d4248cf40 1 -- 172.21.15.167:0/123587811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437400 msgr2=0x55e3fa71c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/123587811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437400 0x55e3fa71c580 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x55e3fa717920 tx=0x55e3fa53b920 comp rx=0 tx=0).stop 2024-09-13T23:43:33.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d4248cf40 1 -- 172.21.15.167:0/123587811 shutdown_connections 2024-09-13T23:43:33.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/123587811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437400 0x55e3fa71c580 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d4248cf40 1 -- 172.21.15.167:0/123587811 >> 172.21.15.167:0/123587811 conn(0x55e3fa436c00 msgr2=0x55e3fa536000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d4248cf40 1 -- 172.21.15.167:0/123587811 shutdown_connections 2024-09-13T23:43:33.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.587+0000 7f4d4248cf40 1 -- 172.21.15.167:0/123587811 wait complete. 2024-09-13T23:43:33.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 Processor -- start 2024-09-13T23:43:33.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 -- start start 2024-09-13T23:43:33.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437000 0x55e3fa71c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fa4c4500 con 0x55e3fa437000 2024-09-13T23:43:33.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d40c45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437000 0x55e3fa71c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d40c45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437000 0x55e3fa71c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52718/0 (socket says 172.21.15.167:52718) 2024-09-13T23:43:33.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d40c45700 1 -- 172.21.15.167:0/3690049536 learned_addr learned my addr 172.21.15.167:0/3690049536 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d40c45700 1 -- 172.21.15.167:0/3690049536 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3fa449ba0 con 0x55e3fa437000 2024-09-13T23:43:33.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d40c45700 1 --2- 172.21.15.167:0/3690049536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437000 0x55e3fa71c580 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55e3fa593e90 tx=0x55e3fa53bb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a6a1f1c90c1856 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d3f442700 1 -- 172.21.15.167:0/3690049536 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa4db1e0 con 0x55e3fa437000 2024-09-13T23:43:33.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d3f442700 1 -- 172.21.15.167:0/3690049536 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e3fa4fbb00 con 0x55e3fa437000 2024-09-13T23:43:33.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d3f442700 1 -- 172.21.15.167:0/3690049536 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa4db4a0 con 0x55e3fa437000 2024-09-13T23:43:33.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 -- 172.21.15.167:0/3690049536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437000 msgr2=0x55e3fa71c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/3690049536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437000 0x55e3fa71c580 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x55e3fa593e90 tx=0x55e3fa53bb60 comp rx=0 tx=0).stop 2024-09-13T23:43:33.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 -- 172.21.15.167:0/3690049536 shutdown_connections 2024-09-13T23:43:33.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/3690049536 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437000 0x55e3fa71c580 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 -- 172.21.15.167:0/3690049536 >> 172.21.15.167:0/3690049536 conn(0x55e3fa436c00 msgr2=0x55e3fa537000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 -- 172.21.15.167:0/3690049536 shutdown_connections 2024-09-13T23:43:33.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 -- 172.21.15.167:0/3690049536 wait complete. 2024-09-13T23:43:33.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 Processor -- start 2024-09-13T23:43:33.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 -- start start 2024-09-13T23:43:33.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437400 0x55e3fa71cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.591+0000 7f4d4248cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fa4c43c0 con 0x55e3fa437400 2024-09-13T23:43:33.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d40c45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437400 0x55e3fa71cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d40c45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437400 0x55e3fa71cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52724/0 (socket says 172.21.15.167:52724) 2024-09-13T23:43:33.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d40c45700 1 -- 172.21.15.167:0/1068693084 learned_addr learned my addr 172.21.15.167:0/1068693084 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d40c45700 1 -- 172.21.15.167:0/1068693084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3fa449a00 con 0x55e3fa437400 2024-09-13T23:43:33.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d40c45700 1 --2- 172.21.15.167:0/1068693084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437400 0x55e3fa71cb00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x55e3fa593f20 tx=0x55e3fa53ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d3dc3f700 1 -- 172.21.15.167:0/1068693084 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa4db600 con 0x55e3fa437400 2024-09-13T23:43:33.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d3dc3f700 1 -- 172.21.15.167:0/1068693084 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e3fa4fbc80 con 0x55e3fa437400 2024-09-13T23:43:33.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1068693084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e3fa449860 con 0x55e3fa437400 2024-09-13T23:43:33.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d3dc3f700 1 -- 172.21.15.167:0/1068693084 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa4db1e0 con 0x55e3fa437400 2024-09-13T23:43:33.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1068693084 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3fa4496c0 con 0x55e3fa437400 2024-09-13T23:43:33.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d4248cf40 1 Processor -- start 2024-09-13T23:43:33.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d4248cf40 1 -- start start 2024-09-13T23:43:33.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d4248cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 0x55e3fa71d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d4248cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fa45c640 con 0x55e3fa437c00 2024-09-13T23:43:33.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d3dc3f700 1 -- 172.21.15.167:0/1068693084 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e3fa530b00 con 0x55e3fa437400 2024-09-13T23:43:33.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d41446700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 0x55e3fa71d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d41446700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 0x55e3fa71d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52738/0 (socket says 172.21.15.167:52738) 2024-09-13T23:43:33.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.595+0000 7f4d41446700 1 -- 172.21.15.167:0/2760369255 learned_addr learned my addr 172.21.15.167:0/2760369255 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d3dc3f700 1 --2- 172.21.15.167:0/1068693084 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa7f8000 0x55e3fa71e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d3dc3f700 1 -- 172.21.15.167:0/1068693084 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e3fa49d180 con 0x55e3fa437400 2024-09-13T23:43:33.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d41446700 1 -- 172.21.15.167:0/2760369255 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3fa448000 con 0x55e3fa437c00 2024-09-13T23:43:33.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d40c45700 1 --2- 172.21.15.167:0/1068693084 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa7f8000 0x55e3fa71e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d41446700 1 --2- 172.21.15.167:0/2760369255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 0x55e3fa71d080 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55e3fa7d2270 tx=0x55e3fa7acde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8443e954fc1e88 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d3b43a700 1 -- 172.21.15.167:0/2760369255 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa4db600 con 0x55e3fa437c00 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d3b43a700 1 -- 172.21.15.167:0/2760369255 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e3fa7f2000 con 0x55e3fa437c00 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d3b43a700 1 -- 172.21.15.167:0/2760369255 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa4db1e0 con 0x55e3fa437c00 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2760369255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 msgr2=0x55e3fa71d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2760369255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 0x55e3fa71d080 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x55e3fa7d2270 tx=0x55e3fa7acde0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d40c45700 1 --2- 172.21.15.167:0/1068693084 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa7f8000 0x55e3fa71e100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55e3fa75b5f0 tx=0x55e3fa7ac240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2760369255 shutdown_connections 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2760369255 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 0x55e3fa71d080 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2760369255 >> 172.21.15.167:0/2760369255 conn(0x55e3fa437800 msgr2=0x55e3fa537800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2760369255 shutdown_connections 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2760369255 wait complete. 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d4248cf40 1 Processor -- start 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d4248cf40 1 -- start start 2024-09-13T23:43:33.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d4248cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 0x55e3fa71d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d4248cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fa45c8c0 con 0x55e3fa437c00 2024-09-13T23:43:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d41446700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 0x55e3fa71d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d41446700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 0x55e3fa71d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52742/0 (socket says 172.21.15.167:52742) 2024-09-13T23:43:33.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 learned_addr learned my addr 172.21.15.167:0/2173901096 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.599+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3fa448ea0 con 0x55e3fa437c00 2024-09-13T23:43:33.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d41446700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 0x55e3fa71d080 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55e3fa75a240 tx=0x55e3fa7ac840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d39c37700 1 -- 172.21.15.167:0/2173901096 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa4dba20 con 0x55e3fa437c00 2024-09-13T23:43:33.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d39c37700 1 -- 172.21.15.167:0/2173901096 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e3fa7f2180 con 0x55e3fa437c00 2024-09-13T23:43:33.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e3fa448820 con 0x55e3fa437c00 2024-09-13T23:43:33.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d39c37700 1 -- 172.21.15.167:0/2173901096 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa4db600 con 0x55e3fa437c00 2024-09-13T23:43:33.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3fa4489c0 con 0x55e3fa437c00 2024-09-13T23:43:33.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d39c37700 1 -- 172.21.15.167:0/2173901096 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e3fa530dc0 con 0x55e3fa437c00 2024-09-13T23:43:33.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d39c37700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa7f8c00 0x55e3fa71e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d39c37700 1 -- 172.21.15.167:0/2173901096 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e3fa49d340 con 0x55e3fa437c00 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d40444700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa7f8c00 0x55e3fa71e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e3fa7f9000 0x55e3fa71d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa7f9400 con 0x55e3fa7f9000 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d41446700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e3fa7f9000 0x55e3fa71d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d40444700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa7f8c00 0x55e3fa71e680 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55e3fa717920 tx=0x55e3fa53b800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.603+0000 7f4d41446700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e3fa7f9000 0x55e3fa71d600 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 20 realm 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa7f9800 con 0x55e3fa7f9000 2024-09-13T23:43:33.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:33.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa7f9c00 con 0x55e3fa7f9000 2024-09-13T23:43:33.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e3fa7f9400 0x55e3fa71db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa7f9800 con 0x55e3fa7f9400 2024-09-13T23:43:33.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d40c45700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e3fa7f9400 0x55e3fa71db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d40c45700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e3fa7f9400 0x55e3fa71db80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:33.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa7f9c00 con 0x55e3fa7f9400 2024-09-13T23:43:33.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:33.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 20 searching for the correct realm 2024-09-13T23:43:33.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e3fa7f9800 0x55e3fa71ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa7f9c00 con 0x55e3fa7f9800 2024-09-13T23:43:33.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.607+0000 7f4d40444700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e3fa7f9800 0x55e3fa71ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d40444700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e3fa7f9800 0x55e3fa71ec00 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa7f9c00 con 0x55e3fa7f9400 2024-09-13T23:43:33.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d40c45700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e3fa7f9c00 0x55e3fa71c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9c00 2024-09-13T23:43:33.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d41446700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e3fa7f9c00 0x55e3fa71c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d41446700 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e3fa7f9c00 0x55e3fa71c580 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9000 2024-09-13T23:43:33.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9000 2024-09-13T23:43:33.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.611+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9800 2024-09-13T23:43:33.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9000 2024-09-13T23:43:33.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9400 2024-09-13T23:43:33.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9400 2024-09-13T23:43:33.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9000 2024-09-13T23:43:33.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9c00 2024-09-13T23:43:33.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9000 2024-09-13T23:43:33.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.615+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9c00 2024-09-13T23:43:33.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9400 2024-09-13T23:43:33.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9800 2024-09-13T23:43:33.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9000 2024-09-13T23:43:33.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9400 2024-09-13T23:43:33.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9800 2024-09-13T23:43:33.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.619+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9000 2024-09-13T23:43:33.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9000 2024-09-13T23:43:33.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9000 2024-09-13T23:43:33.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9800 2024-09-13T23:43:33.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9c00 2024-09-13T23:43:33.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9000 2024-09-13T23:43:33.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9c00 2024-09-13T23:43:33.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.623+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9800 2024-09-13T23:43:33.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.627+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.627+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9c00 2024-09-13T23:43:33.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.627+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.627+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9400 2024-09-13T23:43:33.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.627+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.627+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9400 2024-09-13T23:43:33.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.627+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.627+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9800 2024-09-13T23:43:33.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.627+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.627+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9400 2024-09-13T23:43:33.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.627+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.627+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9400 2024-09-13T23:43:33.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:33.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:33.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:33.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:33.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa800 con 0x55e3fa7f9000 2024-09-13T23:43:33.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aac00 con 0x55e3fa7f9400 2024-09-13T23:43:33.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:33.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8ab000 con 0x55e3fa7f9400 2024-09-13T23:43:33.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:33.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 zone default found 2024-09-13T23:43:33.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 4 Realm: () 2024-09-13T23:43:33.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:33.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:33.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:33.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 zonegroup default 2024-09-13T23:43:33.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8ab400 con 0x55e3fa7f9c00 2024-09-13T23:43:33.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.631+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8ab800 con 0x55e3fa7f9800 2024-09-13T23:43:33.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.635+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.635+0000 7f4d4248cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.635+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.635+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8abc00 con 0x55e3fa7f9800 2024-09-13T23:43:33.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.635+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.635+0000 7f4d4248cf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.635+0000 7f4d4248cf40 20 started sync module instance, tier type = 2024-09-13T23:43:33.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.635+0000 7f4d4248cf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:33.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.635+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8ab800 con 0x55e3fa7f9400 2024-09-13T23:43:33.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.635+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.639+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94437647661824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8abc00 con 0x55e3fa7f9400 2024-09-13T23:43:33.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.639+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa400 con 0x55e3fa7f9000 2024-09-13T23:43:33.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.639+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94437647661824] v33'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.639+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.639+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94437647662976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9000 2024-09-13T23:43:33.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.639+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aac00 con 0x55e3fa7f9400 2024-09-13T23:43:33.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.643+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94437647662976] v33'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.643+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.643+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94437647664128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8ab000 con 0x55e3fa7f9400 2024-09-13T23:43:33.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.643+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa800 con 0x55e3fa7f9c00 2024-09-13T23:43:33.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.647+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94437647664128] v33'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.647+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.647+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94437647665280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8ab400 con 0x55e3fa7f9c00 2024-09-13T23:43:33.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.647+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa400 con 0x55e3fa7f9400 2024-09-13T23:43:33.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.647+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.647+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94437647666432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa000 con 0x55e3fa7f9400 2024-09-13T23:43:33.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.647+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8ab800 con 0x55e3fa7f9800 2024-09-13T23:43:33.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.647+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94437647665280] v33'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.651+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94437647666432] v33'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.651+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.651+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94437647667584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8abc00 con 0x55e3fa7f9800 2024-09-13T23:43:33.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.651+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aac00 con 0x55e3fa7f9800 2024-09-13T23:43:33.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.651+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94437647668736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8ab000 con 0x55e3fa7f9800 2024-09-13T23:43:33.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aa400 con 0x55e3fa7f9c00 2024-09-13T23:43:33.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94437647667584] v33'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'138 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94437647669888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8ab800 con 0x55e3fa7f9c00 2024-09-13T23:43:33.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d4248cf40 20 add_watcher() i=0 2024-09-13T23:43:33.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d4248cf40 20 add_watcher() i=1 2024-09-13T23:43:33.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d4248cf40 20 add_watcher() i=2 2024-09-13T23:43:33.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d4248cf40 20 add_watcher() i=3 2024-09-13T23:43:33.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d4248cf40 20 add_watcher() i=4 2024-09-13T23:43:33.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d4248cf40 20 add_watcher() i=5 2024-09-13T23:43:33.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94437647668736] v33'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d4248cf40 20 add_watcher() i=6 2024-09-13T23:43:33.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.655+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94437647669888] v33'139 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.659+0000 7f4d4248cf40 20 add_watcher() i=7 2024-09-13T23:43:33.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.659+0000 7f4d4248cf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:33.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.659+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8abc00 con 0x55e3fa7f9000 2024-09-13T23:43:33.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.659+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.659+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94437649391616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aac00 con 0x55e3fa7f9000 2024-09-13T23:43:33.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.659+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94437649391616] v33'132 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.659+0000 7f4d27c13700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:33.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.659+0000 7f4d4248cf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:33.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.659+0000 7f4d4248cf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:33.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.659+0000 7f4d4248cf40 1 Processor -- start 2024-09-13T23:43:33.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.659+0000 7f4d4248cf40 1 -- start start 2024-09-13T23:43:33.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 0x55e3fa71f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fa4c43c0 con 0x55e3fa8ab400 2024-09-13T23:43:33.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d40444700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 0x55e3fa71f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d40444700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 0x55e3fa71f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52754/0 (socket says 172.21.15.167:52754) 2024-09-13T23:43:33.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d40444700 1 -- 172.21.15.167:0/1089123483 learned_addr learned my addr 172.21.15.167:0/1089123483 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d40444700 1 -- 172.21.15.167:0/1089123483 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3fa448b60 con 0x55e3fa8ab400 2024-09-13T23:43:33.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d40444700 1 --2- 172.21.15.167:0/1089123483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 0x55e3fa71f180 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55e3fa851740 tx=0x55e3fa53b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22064c508edfe6f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d27412700 1 -- 172.21.15.167:0/1089123483 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa4db760 con 0x55e3fa8ab400 2024-09-13T23:43:33.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d27412700 1 -- 172.21.15.167:0/1089123483 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e3fa7f2600 con 0x55e3fa8ab400 2024-09-13T23:43:33.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d27412700 1 -- 172.21.15.167:0/1089123483 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa8ce000 con 0x55e3fa8ab400 2024-09-13T23:43:33.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1089123483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 msgr2=0x55e3fa71f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/1089123483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 0x55e3fa71f180 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x55e3fa851740 tx=0x55e3fa53b860 comp rx=0 tx=0).stop 2024-09-13T23:43:33.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1089123483 shutdown_connections 2024-09-13T23:43:33.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/1089123483 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 0x55e3fa71f180 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1089123483 >> 172.21.15.167:0/1089123483 conn(0x55e3fa8aa800 msgr2=0x55e3fa538000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1089123483 shutdown_connections 2024-09-13T23:43:33.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1089123483 wait complete. 2024-09-13T23:43:33.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 Processor -- start 2024-09-13T23:43:33.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 -- start start 2024-09-13T23:43:33.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 0x55e3fa71f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.663+0000 7f4d4248cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3fa4c4500 con 0x55e3fa8ab400 2024-09-13T23:43:33.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d40444700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 0x55e3fa71f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d40444700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 0x55e3fa71f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52768/0 (socket says 172.21.15.167:52768) 2024-09-13T23:43:33.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d40444700 1 -- 172.21.15.167:0/2036484430 learned_addr learned my addr 172.21.15.167:0/2036484430 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d40444700 1 -- 172.21.15.167:0/2036484430 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3fa448d00 con 0x55e3fa8ab400 2024-09-13T23:43:33.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d40444700 1 --2- 172.21.15.167:0/2036484430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 0x55e3fa71f180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55e3fa8be7b0 tx=0x55e3fa8c4300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d25c0f700 1 -- 172.21.15.167:0/2036484430 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa8ce160 con 0x55e3fa8ab400 2024-09-13T23:43:33.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d25c0f700 1 -- 172.21.15.167:0/2036484430 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e3fa7f2300 con 0x55e3fa8ab400 2024-09-13T23:43:33.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2036484430 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e3fa448340 con 0x55e3fa8ab400 2024-09-13T23:43:33.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d25c0f700 1 -- 172.21.15.167:0/2036484430 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e3fa8ce2c0 con 0x55e3fa8ab400 2024-09-13T23:43:33.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2036484430 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3fa449d40 con 0x55e3fa8ab400 2024-09-13T23:43:33.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d25c0f700 1 -- 172.21.15.167:0/2036484430 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e3fa531b80 con 0x55e3fa8ab400 2024-09-13T23:43:33.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d25c0f700 1 --2- 172.21.15.167:0/2036484430 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa940000 0x55e3fa946000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d25c0f700 1 -- 172.21.15.167:0/2036484430 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55e3fa49d500 con 0x55e3fa8ab400 2024-09-13T23:43:33.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d41446700 1 --2- 172.21.15.167:0/2036484430 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa940000 0x55e3fa946000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.667+0000 7f4d4248cf40 5 note: GC not initialized 2024-09-13T23:43:33.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d41446700 1 --2- 172.21.15.167:0/2036484430 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa940000 0x55e3fa946000 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55e3fa851cb0 tx=0x55e3fa7ad7a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d2340a700 20 reqs_thread_entry: start 2024-09-13T23:43:33.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:33.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:33.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d21406700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:04 2024 2024-09-13T23:43:33.680 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:33.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d20be4700 20 reqs_thread_entry: start 2024-09-13T23:43:33.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:33.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8abc00 con 0x55e3fa7f9000 2024-09-13T23:43:33.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:33.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:33.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:33.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:33.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 10 cache get: name=default.rgw.meta+root+testx/myfoo : miss 2024-09-13T23:43:33.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa940400 con 0x55e3fa7f9000 2024-09-13T23:43:33.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(63 testx/myfoo [call out=48b,read 0~193 out=193b] v0'0 uv1 ondisk = 0) v8 ==== 197+0+241 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 20 rados_obj.operate() r=0 bl.length=193 2024-09-13T23:43:33.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x11 2024-09-13T23:43:33.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-09-13T23:43:33.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 10 cache get: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 : miss 2024-09-13T23:43:33.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.671+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 8.16 8:6e0d8e78:root::.bucket.meta.testx%3amyfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa940800 con 0x55e3fa7f9800 2024-09-13T23:43:33.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.675+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(64 .bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~371 out=371b,getxattrs out=298b] v0'0 uv1 ondisk = 0) v8 ==== 338+0+733 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.675+0000 7f4d4248cf40 20 rados_obj.operate() r=0 bl.length=371 2024-09-13T23:43:33.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.675+0000 7f4d4248cf40 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:33.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.675+0000 7f4d4248cf40 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:33.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.675+0000 7f4d4248cf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:33.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.675+0000 7f4d4248cf40 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-09-13T23:43:33.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.675+0000 7f4d4248cf40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 2024-09-13T23:43:33.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.675+0000 7f4d4248cf40 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-09-13T23:43:33.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.675+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa940c00 con 0x55e3fa7f9000 2024-09-13T23:43:33.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.675+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(65 testx$tenanteduser.buckets [call] v33'2 uv2 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.675+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:66 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [create,call version.inc in=38b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa941000 con 0x55e3fa7f9800 2024-09-13T23:43:33.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(66 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [create,call,writefull 0~352,setxattr (123)] v33'5 uv5 ondisk = 0) v8 ==== 332+0+0 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d4248cf40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x7 2024-09-13T23:43:33.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d4248cf40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:33.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d4248cf40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:33.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d4248cf40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1, ofs0, ns] 2024-09-13T23:43:33.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94437649393920 in=713b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa941400 con 0x55e3fa7f9400 2024-09-13T23:43:33.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94437647661824 notify 141733920793 ret 0) v3 ==== 743+0+0 (crc 0 0 0) 0x55e3fa7aa820 con 0x55e3fa7f9400 2024-09-13T23:43:33.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d3ac39700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920793 cookie 94437647661824 notifier 4598 bl.length()=701 2024-09-13T23:43:33.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.0 [notify cookie 94437649393920 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d3ac39700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x7 2024-09-13T23:43:33.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d3ac39700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:33.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d3ac39700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:33.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d3ac39700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa9b6000 con 0x55e3fa7f9400 2024-09-13T23:43:33.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== watch-notify(notify_complete (2) cookie 94437649393920 notify 141733920793 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e3fa7aa9c0 con 0x55e3fa7f9400 2024-09-13T23:43:33.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.679+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.683+0000 7f4d4248cf40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): orig_sources= new_sources= 2024-09-13T23:43:33.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.683+0000 7f4d4248cf40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): potential sources added=[] removed=[] 2024-09-13T23:43:33.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.683+0000 7f4d4248cf40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): orig_dests= new_dests= 2024-09-13T23:43:33.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.683+0000 7f4d4248cf40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): potential dests added=[] removed=[] 2024-09-13T23:43:33.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.683+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa941800 con 0x55e3fa7f9c00 2024-09-13T23:43:33.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.683+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(69 foo.buckets [call] v33'26 uv26 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.683+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa941c00 con 0x55e3fa7f9c00 2024-09-13T23:43:33.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v33'27 uv27 ondisk = 0) v8 ==== 275+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d4248cf40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-09-13T23:43:33.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d4248cf40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:33.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d4248cf40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-13T23:43:33.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94437649393920 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa9b6800 con 0x55e3fa7f9800 2024-09-13T23:43:33.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== watch-notify(notify (1) cookie 94437647668736 notify 141733920777 ret 0) v3 ==== 356+0+0 (crc 0 0 0) 0x55e3fa90a9c0 con 0x55e3fa7f9800 2024-09-13T23:43:33.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d3ac39700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920777 cookie 94437647668736 notifier 4598 bl.length()=314 2024-09-13T23:43:33.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d3ac39700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-09-13T23:43:33.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d3ac39700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:33.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(71 notify.6 [notify cookie 94437649393920 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d3ac39700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa9b6400 con 0x55e3fa7f9800 2024-09-13T23:43:33.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== watch-notify(notify_complete (2) cookie 94437649393920 notify 141733920777 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e3fa90ab60 con 0x55e3fa7f9800 2024-09-13T23:43:33.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.687+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa9b6c00 con 0x55e3fa7f9000 2024-09-13T23:43:33.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.691+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(73 testx/myfoo [call,delete] v33'2 uv2 ondisk = 0) v8 ==== 197+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.691+0000 7f4d4248cf40 10 removing default.rgw.meta+root+testx/myfoo from cache 2024-09-13T23:43:33.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.691+0000 7f4d4248cf40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-09-13T23:43:33.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.691+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [notify cookie 94437649393920 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa9b7000 con 0x55e3fa7f9400 2024-09-13T23:43:33.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.691+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== watch-notify(notify (1) cookie 94437647666432 notify 141733920794 ret 0) v3 ==== 197+0+0 (crc 0 0 0) 0x55e3fa7aa9c0 con 0x55e3fa7f9400 2024-09-13T23:43:33.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.691+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(74 notify.4 [notify cookie 94437649393920 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.691+0000 7f4d3ac39700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920794 cookie 94437647666432 notifier 4598 bl.length()=155 2024-09-13T23:43:33.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.691+0000 7f4d3ac39700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa9b6800 con 0x55e3fa7f9400 2024-09-13T23:43:33.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.691+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(75 notify.4 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.691+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== watch-notify(notify_complete (2) cookie 94437649393920 notify 141733920794 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e3fa7aab60 con 0x55e3fa7f9400 2024-09-13T23:43:33.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.691+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 8.16 8:6e0d8e78:root::.bucket.meta.testx%3amyfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa9b7400 con 0x55e3fa7f9800 2024-09-13T23:43:33.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.695+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(76 .bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call,delete] v33'2 uv2 ondisk = 0) v8 ==== 254+0+0 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.695+0000 7f4d4248cf40 10 removing default.rgw.meta+root+.bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 from cache 2024-09-13T23:43:33.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.695+0000 7f4d4248cf40 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1, ofs0, ns] 2024-09-13T23:43:33.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.695+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [notify cookie 94437649393920 in=224b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa9b7800 con 0x55e3fa7f9c00 2024-09-13T23:43:33.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.695+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== watch-notify(notify (1) cookie 94437647669888 notify 141733920787 ret 0) v3 ==== 254+0+0 (crc 0 0 0) 0x55e3fa84e820 con 0x55e3fa7f9c00 2024-09-13T23:43:33.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.695+0000 7f4d3ac39700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920787 cookie 94437647669888 notifier 4598 bl.length()=212 2024-09-13T23:43:33.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.695+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(77 notify.7 [notify cookie 94437649393920 out=8b] v0'0 uv138 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.695+0000 7f4d3ac39700 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa940800 con 0x55e3fa7f9c00 2024-09-13T23:43:33.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.695+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== watch-notify(notify_complete (2) cookie 94437649393920 notify 141733920787 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e3fa84e9c0 con 0x55e3fa7f9c00 2024-09-13T23:43:33.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.695+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(78 notify.7 [notify-ack] v0'0 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.699+0000 7f4d27c13700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:33.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.699+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94437649391616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa9b7c00 con 0x55e3fa7f9000 2024-09-13T23:43:33.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.699+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94437649391616] v33'133 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.699+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94437647661824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa940400 con 0x55e3fa7f9400 2024-09-13T23:43:33.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.703+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94437647661824] v33'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.703+0000 7f4d4248cf40 20 remove_watcher() i=0 2024-09-13T23:43:33.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.703+0000 7f4d4248cf40 2 removed watcher, disabling cache 2024-09-13T23:43:33.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.703+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94437647662976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa940c00 con 0x55e3fa7f9000 2024-09-13T23:43:33.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.707+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94437647662976] v33'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9000 2024-09-13T23:43:33.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.707+0000 7f4d4248cf40 20 remove_watcher() i=1 2024-09-13T23:43:33.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.707+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94437647664128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa941800 con 0x55e3fa7f9400 2024-09-13T23:43:33.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.707+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94437647664128] v33'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.707+0000 7f4d4248cf40 20 remove_watcher() i=2 2024-09-13T23:43:33.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.707+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94437647665280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa941c00 con 0x55e3fa7f9c00 2024-09-13T23:43:33.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.711+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 18 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94437647665280] v33'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.711+0000 7f4d4248cf40 20 remove_watcher() i=3 2024-09-13T23:43:33.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.711+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94437647666432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8aac00 con 0x55e3fa7f9400 2024-09-13T23:43:33.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.715+0000 7f4d40c45700 1 -- 172.21.15.167:0/2173901096 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94437647666432] v33'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b200 con 0x55e3fa7f9400 2024-09-13T23:43:33.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.715+0000 7f4d4248cf40 20 remove_watcher() i=4 2024-09-13T23:43:33.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.715+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94437647667584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa8abc00 con 0x55e3fa7f9800 2024-09-13T23:43:33.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.715+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94437647667584] v33'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.715+0000 7f4d4248cf40 20 remove_watcher() i=5 2024-09-13T23:43:33.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.715+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94437647668736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa9b6000 con 0x55e3fa7f9800 2024-09-13T23:43:33.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94437647668736] v33'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89b440 con 0x55e3fa7f9800 2024-09-13T23:43:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d4248cf40 20 remove_watcher() i=6 2024-09-13T23:43:33.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94437647669888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55e3fa9b7000 con 0x55e3fa7f9c00 2024-09-13T23:43:33.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d41446700 1 -- 172.21.15.167:0/2173901096 <== osd.2 v2:172.21.15.167:6808/1180369139 19 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94437647669888] v33'140 uv138 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e3fa89ab40 con 0x55e3fa7f9c00 2024-09-13T23:43:33.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d4248cf40 20 remove_watcher() i=7 2024-09-13T23:43:33.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e3fa7f9400 msgr2=0x55e3fa71db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e3fa7f9400 0x55e3fa71db80 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e3fa7f9000 msgr2=0x55e3fa71d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e3fa7f9000 0x55e3fa71d600 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e3fa7f9c00 msgr2=0x55e3fa71c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e3fa7f9c00 0x55e3fa71c580 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e3fa7f9800 msgr2=0x55e3fa71ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.719+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e3fa7f9800 0x55e3fa71ec00 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa7f8c00 msgr2=0x55e3fa71e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa7f8c00 0x55e3fa71e680 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55e3fa717920 tx=0x55e3fa53b800 comp rx=0 tx=0).stop 2024-09-13T23:43:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 msgr2=0x55e3fa71d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 0x55e3fa71d080 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55e3fa75a240 tx=0x55e3fa7ac840 comp rx=0 tx=0).stop 2024-09-13T23:43:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d40444700 1 -- 172.21.15.167:0/2173901096 reap_dead start 2024-09-13T23:43:33.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 shutdown_connections 2024-09-13T23:43:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2173901096 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437c00 0x55e3fa71d080 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 >> 172.21.15.167:0/2173901096 conn(0x55e3fa437800 msgr2=0x55e3fa537800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 shutdown_connections 2024-09-13T23:43:33.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2173901096 wait complete. 2024-09-13T23:43:33.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2036484430 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa940000 msgr2=0x55e3fa946000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2036484430 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa940000 0x55e3fa946000 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x55e3fa851cb0 tx=0x55e3fa7ad7a0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2036484430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 msgr2=0x55e3fa71f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2036484430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 0x55e3fa71f180 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x55e3fa8be7b0 tx=0x55e3fa8c4300 comp rx=0 tx=0).stop 2024-09-13T23:43:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2036484430 shutdown_connections 2024-09-13T23:43:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2036484430 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa940000 0x55e3fa946000 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/2036484430 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa8ab400 0x55e3fa71f180 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2036484430 >> 172.21.15.167:0/2036484430 conn(0x55e3fa8aa800 msgr2=0x55e3fa538000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2036484430 shutdown_connections 2024-09-13T23:43:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/2036484430 wait complete. 2024-09-13T23:43:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1068693084 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa7f8000 msgr2=0x55e3fa71e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/1068693084 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa7f8000 0x55e3fa71e100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x55e3fa75b5f0 tx=0x55e3fa7ac240 comp rx=0 tx=0).stop 2024-09-13T23:43:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1068693084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437400 msgr2=0x55e3fa71cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/1068693084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437400 0x55e3fa71cb00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x55e3fa593f20 tx=0x55e3fa53ba40 comp rx=0 tx=0).stop 2024-09-13T23:43:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1068693084 shutdown_connections 2024-09-13T23:43:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/1068693084 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e3fa7f8000 0x55e3fa71e100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 --2- 172.21.15.167:0/1068693084 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e3fa437400 0x55e3fa71cb00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1068693084 >> 172.21.15.167:0/1068693084 conn(0x55e3fa436c00 msgr2=0x55e3fa537000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1068693084 shutdown_connections 2024-09-13T23:43:33.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.723+0000 7f4d4248cf40 1 -- 172.21.15.167:0/1068693084 wait complete. 2024-09-13T23:43:33.739 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-09-13T23:43:33.739 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-09-13T23:43:33.739 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --tenant testx --uid tenanteduser 2024-09-13T23:43:33.781 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:33.781 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:33.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.771+0000 7fc264324f40 1 Processor -- start 2024-09-13T23:43:33.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.771+0000 7fc264324f40 1 -- start start 2024-09-13T23:43:33.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.771+0000 7fc264324f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5619e6601000 0x5619e68e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.771+0000 7fc264324f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5619e66c5500 con 0x5619e6601400 2024-09-13T23:43:33.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.771+0000 7fc264324f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5619e668ea00 con 0x5619e6601000 2024-09-13T23:43:33.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc262add700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5619e6601000 0x5619e68e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc262add700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5619e6601000 0x5619e68e6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52772/0 (socket says 172.21.15.167:52772) 2024-09-13T23:43:33.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc262add700 1 -- 172.21.15.167:0/2908079853 learned_addr learned my addr 172.21.15.167:0/2908079853 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc2622dc700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5619e6601400 0x5619e6700800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45550/0 (socket says 172.21.15.167:45550) 2024-09-13T23:43:33.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc262add700 1 -- 172.21.15.167:0/2908079853 >> v1:172.21.15.12:6789/0 conn(0x5619e6601400 legacy=0x5619e6700800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc261adb700 1 -- 172.21.15.167:0/2908079853 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081519908 0 0) 0x5619e675c000 con 0x5619e6601400 2024-09-13T23:43:33.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc262add700 1 -- 172.21.15.167:0/2908079853 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e6613d40 con 0x5619e6601000 2024-09-13T23:43:33.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc262add700 1 --2- 172.21.15.167:0/2908079853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x5619e68e1a70 tx=0x5619e6705920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ccf592bae9312fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc261adb700 1 -- 172.21.15.167:0/2908079853 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e66a54a0 con 0x5619e6601000 2024-09-13T23:43:33.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc261adb700 1 -- 172.21.15.167:0/2908079853 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619e66c5800 con 0x5619e6601000 2024-09-13T23:43:33.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc261adb700 1 -- 172.21.15.167:0/2908079853 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e66a5600 con 0x5619e6601000 2024-09-13T23:43:33.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc264324f40 1 -- 172.21.15.167:0/2908079853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 msgr2=0x5619e68e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc264324f40 1 --2- 172.21.15.167:0/2908079853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x5619e68e1a70 tx=0x5619e6705920 comp rx=0 tx=0).stop 2024-09-13T23:43:33.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc264324f40 1 -- 172.21.15.167:0/2908079853 shutdown_connections 2024-09-13T23:43:33.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc264324f40 1 --2- 172.21.15.167:0/2908079853 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc264324f40 1 -- 172.21.15.167:0/2908079853 >> 172.21.15.167:0/2908079853 conn(0x5619e6600c00 msgr2=0x5619e6700000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc264324f40 1 -- 172.21.15.167:0/2908079853 shutdown_connections 2024-09-13T23:43:33.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc264324f40 1 -- 172.21.15.167:0/2908079853 wait complete. 2024-09-13T23:43:33.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc264324f40 1 Processor -- start 2024-09-13T23:43:33.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc264324f40 1 -- start start 2024-09-13T23:43:33.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc264324f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.775+0000 7fc264324f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e668e500 con 0x5619e6601000 2024-09-13T23:43:33.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc262add700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc262add700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52780/0 (socket says 172.21.15.167:52780) 2024-09-13T23:43:33.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc262add700 1 -- 172.21.15.167:0/2207900743 learned_addr learned my addr 172.21.15.167:0/2207900743 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc262add700 1 -- 172.21.15.167:0/2207900743 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e6613ba0 con 0x5619e6601000 2024-09-13T23:43:33.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc262add700 1 --2- 172.21.15.167:0/2207900743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x5619e68e1f50 tx=0x5619e6705800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da30433bc45e93a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc2612da700 1 -- 172.21.15.167:0/2207900743 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e66a5760 con 0x5619e6601000 2024-09-13T23:43:33.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc2612da700 1 -- 172.21.15.167:0/2207900743 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619e66c5b00 con 0x5619e6601000 2024-09-13T23:43:33.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc2612da700 1 -- 172.21.15.167:0/2207900743 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e66a54a0 con 0x5619e6601000 2024-09-13T23:43:33.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc264324f40 1 -- 172.21.15.167:0/2207900743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 msgr2=0x5619e68e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc264324f40 1 --2- 172.21.15.167:0/2207900743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x5619e68e1f50 tx=0x5619e6705800 comp rx=0 tx=0).stop 2024-09-13T23:43:33.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc264324f40 1 -- 172.21.15.167:0/2207900743 shutdown_connections 2024-09-13T23:43:33.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc264324f40 1 --2- 172.21.15.167:0/2207900743 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc264324f40 1 -- 172.21.15.167:0/2207900743 >> 172.21.15.167:0/2207900743 conn(0x5619e6600c00 msgr2=0x5619e6701000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc264324f40 1 -- 172.21.15.167:0/2207900743 shutdown_connections 2024-09-13T23:43:33.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc264324f40 1 -- 172.21.15.167:0/2207900743 wait complete. 2024-09-13T23:43:33.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc264324f40 1 Processor -- start 2024-09-13T23:43:33.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc264324f40 1 -- start start 2024-09-13T23:43:33.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc264324f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc264324f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e668e3c0 con 0x5619e6601000 2024-09-13T23:43:33.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc262add700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc262add700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52796/0 (socket says 172.21.15.167:52796) 2024-09-13T23:43:33.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc262add700 1 -- 172.21.15.167:0/3721836200 learned_addr learned my addr 172.21.15.167:0/3721836200 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.779+0000 7fc262add700 1 -- 172.21.15.167:0/3721836200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e6613a00 con 0x5619e6601000 2024-09-13T23:43:33.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc262add700 1 --2- 172.21.15.167:0/3721836200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5619e7461ef0 tx=0x5619e6705920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc25fad7700 1 -- 172.21.15.167:0/3721836200 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e66a5600 con 0x5619e6601000 2024-09-13T23:43:33.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc25fad7700 1 -- 172.21.15.167:0/3721836200 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619e66c5c80 con 0x5619e6601000 2024-09-13T23:43:33.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc264324f40 1 -- 172.21.15.167:0/3721836200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619e6613860 con 0x5619e6601000 2024-09-13T23:43:33.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc25fad7700 1 -- 172.21.15.167:0/3721836200 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e66a58c0 con 0x5619e6601000 2024-09-13T23:43:33.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc264324f40 1 -- 172.21.15.167:0/3721836200 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619e66136c0 con 0x5619e6601000 2024-09-13T23:43:33.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc264324f40 1 Processor -- start 2024-09-13T23:43:33.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc264324f40 1 -- start start 2024-09-13T23:43:33.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc264324f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 0x5619e68e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc264324f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e6626640 con 0x5619e6601800 2024-09-13T23:43:33.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc2632de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 0x5619e68e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc2632de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 0x5619e68e6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52802/0 (socket says 172.21.15.167:52802) 2024-09-13T23:43:33.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc2632de700 1 -- 172.21.15.167:0/4275369929 learned_addr learned my addr 172.21.15.167:0/4275369929 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc25fad7700 1 -- 172.21.15.167:0/3721836200 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5619e66fab00 con 0x5619e6601000 2024-09-13T23:43:33.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc2632de700 1 -- 172.21.15.167:0/4275369929 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e6612000 con 0x5619e6601800 2024-09-13T23:43:33.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc25fad7700 1 --2- 172.21.15.167:0/3721836200 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e693c000 0x5619e68e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc25fad7700 1 -- 172.21.15.167:0/3721836200 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x5619e6667180 con 0x5619e6601000 2024-09-13T23:43:33.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc2632de700 1 --2- 172.21.15.167:0/4275369929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 0x5619e68e6b00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5619e7474f90 tx=0x5619e69628a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9ec91152cc2ba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc262add700 1 --2- 172.21.15.167:0/3721836200 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e693c000 0x5619e68e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc25d2d2700 1 -- 172.21.15.167:0/4275369929 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e66a5600 con 0x5619e6601800 2024-09-13T23:43:33.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc25d2d2700 1 -- 172.21.15.167:0/4275369929 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619e6970000 con 0x5619e6601800 2024-09-13T23:43:33.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.783+0000 7fc25d2d2700 1 -- 172.21.15.167:0/4275369929 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e66a58c0 con 0x5619e6601800 2024-09-13T23:43:33.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 -- 172.21.15.167:0/4275369929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 msgr2=0x5619e68e6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 --2- 172.21.15.167:0/4275369929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 0x5619e68e6b00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x5619e7474f90 tx=0x5619e69628a0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc262add700 1 --2- 172.21.15.167:0/3721836200 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e693c000 0x5619e68e7080 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x5619e7475050 tx=0x5619e6962120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 -- 172.21.15.167:0/4275369929 shutdown_connections 2024-09-13T23:43:33.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 --2- 172.21.15.167:0/4275369929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 0x5619e68e6b00 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 -- 172.21.15.167:0/4275369929 >> 172.21.15.167:0/4275369929 conn(0x5619e6601400 msgr2=0x5619e6701800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 -- 172.21.15.167:0/4275369929 shutdown_connections 2024-09-13T23:43:33.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 -- 172.21.15.167:0/4275369929 wait complete. 2024-09-13T23:43:33.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 Processor -- start 2024-09-13T23:43:33.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 -- start start 2024-09-13T23:43:33.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 0x5619e68e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e66268c0 con 0x5619e6601800 2024-09-13T23:43:33.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc2632de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 0x5619e68e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc2632de700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 0x5619e68e6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52816/0 (socket says 172.21.15.167:52816) 2024-09-13T23:43:33.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 learned_addr learned my addr 172.21.15.167:0/796547858 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e6612ea0 con 0x5619e6601800 2024-09-13T23:43:33.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc2632de700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 0x5619e68e6b00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x5619e7461f50 tx=0x5619e6705da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc25bacf700 1 -- 172.21.15.167:0/796547858 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e66a5760 con 0x5619e6601800 2024-09-13T23:43:33.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619e6612820 con 0x5619e6601800 2024-09-13T23:43:33.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc25bacf700 1 -- 172.21.15.167:0/796547858 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619e6970180 con 0x5619e6601800 2024-09-13T23:43:33.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619e66129c0 con 0x5619e6601800 2024-09-13T23:43:33.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc25bacf700 1 -- 172.21.15.167:0/796547858 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e66a5600 con 0x5619e6601800 2024-09-13T23:43:33.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.787+0000 7fc264324f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc25bacf700 1 -- 172.21.15.167:0/796547858 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5619e66fb080 con 0x5619e6601800 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc25bacf700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e693cc00 0x5619e68e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc25bacf700 1 -- 172.21.15.167:0/796547858 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x5619e6667340 con 0x5619e6601800 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619e693d000 0x5619e68e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d400 con 0x5619e693d000 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc2622dc700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e693cc00 0x5619e68e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc2632de700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619e693d000 0x5619e68e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc2622dc700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e693cc00 0x5619e68e7600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5619e69abe30 tx=0x5619e6963f80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc2632de700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619e693d000 0x5619e68e7b80 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 20 realm 2024-09-13T23:43:33.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d000 2024-09-13T23:43:33.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:33.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693dc00 con 0x5619e693d000 2024-09-13T23:43:33.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619e693d400 0x5619e68e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d400 2024-09-13T23:43:33.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.791+0000 7fc262add700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619e693d400 0x5619e68e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc262add700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619e693d400 0x5619e68e8100 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc264324f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:33.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc264324f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693dc00 con 0x5619e693d400 2024-09-13T23:43:33.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc264324f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:33.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc264324f40 20 searching for the correct realm 2024-09-13T23:43:33.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc264324f40 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619e6a6a000 0x5619e68e8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6a400 con 0x5619e6a6a000 2024-09-13T23:43:33.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc2622dc700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619e6a6a000 0x5619e68e8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc2622dc700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619e6a6a000 0x5619e68e8680 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6a400 con 0x5619e693d400 2024-09-13T23:43:33.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc262add700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619e6a6a400 0x5619e68e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.795+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a400 2024-09-13T23:43:33.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc2632de700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619e6a6a400 0x5619e68e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc2632de700 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619e6a6a400 0x5619e68e8c00 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d000 2024-09-13T23:43:33.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d000 2024-09-13T23:43:33.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a000 2024-09-13T23:43:33.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d000 2024-09-13T23:43:33.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d400 2024-09-13T23:43:33.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.799+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d400 2024-09-13T23:43:33.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d000 2024-09-13T23:43:33.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a400 2024-09-13T23:43:33.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d000 2024-09-13T23:43:33.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a400 2024-09-13T23:43:33.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d400 2024-09-13T23:43:33.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a000 2024-09-13T23:43:33.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.803+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d000 2024-09-13T23:43:33.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d400 2024-09-13T23:43:33.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a000 2024-09-13T23:43:33.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d000 2024-09-13T23:43:33.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d000 2024-09-13T23:43:33.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d000 2024-09-13T23:43:33.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a000 2024-09-13T23:43:33.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.807+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a400 2024-09-13T23:43:33.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.811+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.811+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d000 2024-09-13T23:43:33.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.811+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.811+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a400 2024-09-13T23:43:33.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.811+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.811+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a000 2024-09-13T23:43:33.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.811+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.811+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a400 2024-09-13T23:43:33.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.811+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.811+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d400 2024-09-13T23:43:33.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.811+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.811+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d400 2024-09-13T23:43:33.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a000 2024-09-13T23:43:33.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d400 2024-09-13T23:43:33.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e693d400 2024-09-13T23:43:33.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc264324f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:33.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc264324f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:33.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc264324f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:33.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc264324f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:33.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc264324f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6a800 con 0x5619e693d000 2024-09-13T23:43:33.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc264324f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc264324f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6ac00 con 0x5619e693d400 2024-09-13T23:43:33.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc264324f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:33.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc264324f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.815+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b000 con 0x5619e693d400 2024-09-13T23:43:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 20 zone default found 2024-09-13T23:43:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 4 Realm: () 2024-09-13T23:43:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 20 zonegroup default 2024-09-13T23:43:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b400 con 0x5619e6a6a400 2024-09-13T23:43:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b800 con 0x5619e6a6a000 2024-09-13T23:43:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6bc00 con 0x5619e6a6a000 2024-09-13T23:43:33.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:33.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 20 started sync module instance, tier type = 2024-09-13T23:43:33.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:33.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.819+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b800 con 0x5619e693d400 2024-09-13T23:43:33.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.823+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.823+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94669242227456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6bc00 con 0x5619e693d400 2024-09-13T23:43:33.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.823+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6ac00 con 0x5619e693d000 2024-09-13T23:43:33.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.827+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.827+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94669242227456] v33'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.827+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94669242228608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b000 con 0x5619e693d000 2024-09-13T23:43:33.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.827+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b800 con 0x5619e693d400 2024-09-13T23:43:33.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.827+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94669242228608] v33'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.827+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.827+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94669242229760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693dc00 con 0x5619e693d400 2024-09-13T23:43:33.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.827+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693d800 con 0x5619e6a6a400 2024-09-13T23:43:33.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.831+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.831+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94669242230912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6a800 con 0x5619e6a6a400 2024-09-13T23:43:33.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.831+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b400 con 0x5619e693d400 2024-09-13T23:43:33.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.831+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94669242229760] v33'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.835+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94669242230912] v33'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.835+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.835+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94669242232064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6ac00 con 0x5619e693d400 2024-09-13T23:43:33.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.835+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b000 con 0x5619e6a6a000 2024-09-13T23:43:33.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.835+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.835+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94669242233216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6c000 con 0x5619e6a6a000 2024-09-13T23:43:33.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.835+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6c400 con 0x5619e6a6a000 2024-09-13T23:43:33.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.835+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94669242232064] v33'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94669242233216] v33'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94669242234368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6c800 con 0x5619e6a6a000 2024-09-13T23:43:33.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6cc00 con 0x5619e6a6a400 2024-09-13T23:43:33.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'141 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94669242234368] v33'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94669242235520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6d000 con 0x5619e6a6a400 2024-09-13T23:43:33.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc264324f40 20 add_watcher() i=0 2024-09-13T23:43:33.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc264324f40 20 add_watcher() i=1 2024-09-13T23:43:33.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc264324f40 20 add_watcher() i=2 2024-09-13T23:43:33.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc264324f40 20 add_watcher() i=3 2024-09-13T23:43:33.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc264324f40 20 add_watcher() i=4 2024-09-13T23:43:33.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc264324f40 20 add_watcher() i=5 2024-09-13T23:43:33.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.839+0000 7fc264324f40 20 add_watcher() i=6 2024-09-13T23:43:33.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.843+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94669242235520] v33'142 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.843+0000 7fc264324f40 20 add_watcher() i=7 2024-09-13T23:43:33.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.843+0000 7fc264324f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:33.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.843+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6d400 con 0x5619e693d000 2024-09-13T23:43:33.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.843+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.843+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94669243883520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6d800 con 0x5619e693d000 2024-09-13T23:43:33.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94669243883520] v33'134 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc249aab700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:33.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc264324f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:33.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc264324f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:33.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc264324f40 1 Processor -- start 2024-09-13T23:43:33.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc264324f40 1 -- start start 2024-09-13T23:43:33.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc264324f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 0x5619e68e9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc264324f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e668e3c0 con 0x5619e6a6cc00 2024-09-13T23:43:33.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc2622dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 0x5619e68e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc2622dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 0x5619e68e9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52830/0 (socket says 172.21.15.167:52830) 2024-09-13T23:43:33.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc2622dc700 1 -- 172.21.15.167:0/3481151771 learned_addr learned my addr 172.21.15.167:0/3481151771 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc2622dc700 1 -- 172.21.15.167:0/3481151771 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e6612b60 con 0x5619e6a6cc00 2024-09-13T23:43:33.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc2622dc700 1 --2- 172.21.15.167:0/3481151771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 0x5619e68e9180 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x5619e6a7c090 tx=0x5619e6a7e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8ae90e8b6c34c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc2492aa700 1 -- 172.21.15.167:0/3481151771 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e6a86000 con 0x5619e6a6cc00 2024-09-13T23:43:33.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc2492aa700 1 -- 172.21.15.167:0/3481151771 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619e6970600 con 0x5619e6a6cc00 2024-09-13T23:43:33.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.847+0000 7fc2492aa700 1 -- 172.21.15.167:0/3481151771 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e6a86160 con 0x5619e6a6cc00 2024-09-13T23:43:33.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 -- 172.21.15.167:0/3481151771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 msgr2=0x5619e68e9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 --2- 172.21.15.167:0/3481151771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 0x5619e68e9180 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x5619e6a7c090 tx=0x5619e6a7e000 comp rx=0 tx=0).stop 2024-09-13T23:43:33.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 -- 172.21.15.167:0/3481151771 shutdown_connections 2024-09-13T23:43:33.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 --2- 172.21.15.167:0/3481151771 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 0x5619e68e9180 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 -- 172.21.15.167:0/3481151771 >> 172.21.15.167:0/3481151771 conn(0x5619e6a6dc00 msgr2=0x5619e6702000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 -- 172.21.15.167:0/3481151771 shutdown_connections 2024-09-13T23:43:33.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 -- 172.21.15.167:0/3481151771 wait complete. 2024-09-13T23:43:33.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 Processor -- start 2024-09-13T23:43:33.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 -- start start 2024-09-13T23:43:33.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 0x5619e68e9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619e668e500 con 0x5619e6a6cc00 2024-09-13T23:43:33.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc2622dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 0x5619e68e9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc2622dc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 0x5619e68e9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52842/0 (socket says 172.21.15.167:52842) 2024-09-13T23:43:33.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc2622dc700 1 -- 172.21.15.167:0/2581674119 learned_addr learned my addr 172.21.15.167:0/2581674119 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:33.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc2622dc700 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619e6612d00 con 0x5619e6a6cc00 2024-09-13T23:43:33.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc2622dc700 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 0x5619e68e9180 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x5619e6a7d350 tx=0x5619e6a7e2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc247aa7700 1 -- 172.21.15.167:0/2581674119 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e6a862c0 con 0x5619e6a6cc00 2024-09-13T23:43:33.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc247aa7700 1 -- 172.21.15.167:0/2581674119 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619e6970300 con 0x5619e6a6cc00 2024-09-13T23:43:33.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619e6612340 con 0x5619e6a6cc00 2024-09-13T23:43:33.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc247aa7700 1 -- 172.21.15.167:0/2581674119 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619e6a86000 con 0x5619e6a6cc00 2024-09-13T23:43:33.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.851+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619e6612b60 con 0x5619e6a6cc00 2024-09-13T23:43:33.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.855+0000 7fc247aa7700 1 -- 172.21.15.167:0/2581674119 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5619e66fbb80 con 0x5619e6a6cc00 2024-09-13T23:43:33.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.855+0000 7fc247aa7700 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e6b02000 0x5619e6b08000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.855+0000 7fc247aa7700 1 -- 172.21.15.167:0/2581674119 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x5619e6667500 con 0x5619e6a6cc00 2024-09-13T23:43:33.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.855+0000 7fc2632de700 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e6b02000 0x5619e6b08000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.855+0000 7fc264324f40 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619e6a6d000 0x5619e68e9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.855+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6d400 con 0x5619e6a6d000 2024-09-13T23:43:33.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.855+0000 7fc262add700 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619e6a6d000 0x5619e68e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.855+0000 7fc2632de700 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e6b02000 0x5619e6b08000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x5619e74751d0 tx=0x5619e69633e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.855+0000 7fc262add700 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619e6a6d000 0x5619e68e9700 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.859+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'28 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.859+0000 7fc264324f40 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619e6a6c000 0x5619e6b08580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.859+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6c400 con 0x5619e6a6c000 2024-09-13T23:43:33.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.859+0000 7fc2622dc700 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619e6a6c000 0x5619e6b08580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.859+0000 7fc2622dc700 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619e6a6c000 0x5619e6b08580 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.859+0000 7fc2622dc700 1 -- 172.21.15.167:0/2581674119 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'16 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6c000 2024-09-13T23:43:33.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.859+0000 7fc264324f40 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619e6a6a800 0x5619e6b08b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.859+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b000 con 0x5619e6a6a800 2024-09-13T23:43:33.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.859+0000 7fc2632de700 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619e6a6a800 0x5619e6b08b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.863+0000 7fc2632de700 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619e6a6a800 0x5619e6b08b00 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.863+0000 7fc2632de700 1 -- 172.21.15.167:0/2581674119 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a800 2024-09-13T23:43:33.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.863+0000 7fc264324f40 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619e693d800 0x5619e6b09080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:33.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.863+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b02400 con 0x5619e693d800 2024-09-13T23:43:33.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.863+0000 7fc262add700 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619e693d800 0x5619e6b09080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:33.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.863+0000 7fc262add700 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619e693d800 0x5619e6b09080 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:33.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.867+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'135 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d800 2024-09-13T23:43:33.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.867+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b02800 con 0x5619e6a6d000 2024-09-13T23:43:33.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.871+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'38 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.871+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b02c00 con 0x5619e6a6a800 2024-09-13T23:43:33.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.871+0000 7fc2632de700 1 -- 172.21.15.167:0/2581674119 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'28 uv10 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a800 2024-09-13T23:43:33.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.871+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03000 con 0x5619e6a6d000 2024-09-13T23:43:33.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.875+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'18 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.875+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03400 con 0x5619e693d800 2024-09-13T23:43:33.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.879+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'26 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d800 2024-09-13T23:43:33.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.879+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03800 con 0x5619e6a6d000 2024-09-13T23:43:33.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.879+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.879+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03c00 con 0x5619e6a6d000 2024-09-13T23:43:33.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.883+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'23 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.883+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6d400 con 0x5619e6a6d000 2024-09-13T23:43:33.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.887+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'39 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.887+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6bc00 con 0x5619e6a6a800 2024-09-13T23:43:33.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.887+0000 7fc2632de700 1 -- 172.21.15.167:0/2581674119 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'18 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a800 2024-09-13T23:43:33.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.887+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b800 con 0x5619e6a6d000 2024-09-13T23:43:33.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.891+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'16 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.891+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b400 con 0x5619e693d800 2024-09-13T23:43:33.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.895+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'31 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d800 2024-09-13T23:43:33.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.895+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6ac00 con 0x5619e693d800 2024-09-13T23:43:33.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.895+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'136 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d800 2024-09-13T23:43:33.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.895+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693dc00 con 0x5619e6a6d000 2024-09-13T23:43:33.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.899+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'32 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.899+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b02400 con 0x5619e6a6d000 2024-09-13T23:43:33.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.903+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'40 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.903+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b02800 con 0x5619e6a6a800 2024-09-13T23:43:33.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.903+0000 7fc2632de700 1 -- 172.21.15.167:0/2581674119 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'16 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a800 2024-09-13T23:43:33.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.903+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03000 con 0x5619e6a6d000 2024-09-13T23:43:33.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.907+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'24 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.907+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03400 con 0x5619e693d800 2024-09-13T23:43:33.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.911+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'137 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d800 2024-09-13T23:43:33.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.911+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03c00 con 0x5619e6a6d000 2024-09-13T23:43:33.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.911+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'33 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.911+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03800 con 0x5619e6a6d000 2024-09-13T23:43:33.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.915+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'26 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.915+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b800 con 0x5619e6a6a800 2024-09-13T23:43:33.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.919+0000 7fc2632de700 1 -- 172.21.15.167:0/2581674119 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a800 2024-09-13T23:43:33.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.919+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6d400 con 0x5619e6a6d000 2024-09-13T23:43:33.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.919+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.919+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b02c00 con 0x5619e6a6a800 2024-09-13T23:43:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.923+0000 7fc2632de700 1 -- 172.21.15.167:0/2581674119 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'12 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a800 2024-09-13T23:43:33.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.923+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b02800 con 0x5619e6a6a800 2024-09-13T23:43:33.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.927+0000 7fc2632de700 1 -- 172.21.15.167:0/2581674119 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'29 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a800 2024-09-13T23:43:33.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.927+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b02400 con 0x5619e6a6a800 2024-09-13T23:43:33.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.927+0000 7fc2632de700 1 -- 172.21.15.167:0/2581674119 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a800 2024-09-13T23:43:33.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.927+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03000 con 0x5619e6a6d000 2024-09-13T23:43:33.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.931+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'16 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.931+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03400 con 0x5619e693d800 2024-09-13T23:43:33.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.935+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'27 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d800 2024-09-13T23:43:33.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.935+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03c00 con 0x5619e6a6d000 2024-09-13T23:43:33.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.935+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'34 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e6a6d000 2024-09-13T23:43:33.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.935+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03800 con 0x5619e693d800 2024-09-13T23:43:33.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.939+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'32 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d800 2024-09-13T23:43:33.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.939+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693dc00 con 0x5619e6a6c000 2024-09-13T23:43:33.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.943+0000 7fc2622dc700 1 -- 172.21.15.167:0/2581674119 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'22 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6c000 2024-09-13T23:43:33.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.943+0000 7fc2452a2700 20 reqs_thread_entry: start 2024-09-13T23:43:33.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.943+0000 7fc264324f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:33.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.943+0000 7fc264324f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:33.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.943+0000 7fc24329e700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:04 2024 2024-09-13T23:43:33.954 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:33.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.943+0000 7fc264324f40 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-09-13T23:43:33.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.943+0000 7fc264324f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:33.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.943+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b800 con 0x5619e6a6a000 2024-09-13T23:43:33.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.943+0000 7fc242a7c700 20 reqs_thread_entry: start 2024-09-13T23:43:33.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.947+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(62 testx$tenanteduser [call out=48b,read 0~385 out=385b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 246+0+508 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.947+0000 7fc264324f40 20 rados_obj.operate() r=0 bl.length=385 2024-09-13T23:43:33.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.947+0000 7fc264324f40 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-09-13T23:43:33.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.947+0000 7fc264324f40 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-09-13T23:43:33.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.947+0000 7fc264324f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:33.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.947+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b400 con 0x5619e693d000 2024-09-13T23:43:33.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.947+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(63 testx$tenanteduser.buckets [call out=15b] v0'0 uv2 ondisk = 0) v8 ==== 170+0+15 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.947+0000 7fc264324f40 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-09-13T23:43:33.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.947+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6ac00 con 0x5619e6a6a000 2024-09-13T23:43:33.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.947+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(64 p5YnriCv1nAtykxBrupQ [delete] v33'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.947+0000 7fc264324f40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-13T23:43:33.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.947+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [notify cookie 94669243886976 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6d800 con 0x5619e693d400 2024-09-13T23:43:33.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.951+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94669242229760 notify 141733920795 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x5619e69611e0 con 0x5619e693d400 2024-09-13T23:43:33.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.951+0000 7fc25d2d2700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920795 cookie 94669242229760 notifier 4606 bl.length()=170 2024-09-13T23:43:33.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.951+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 notify.2 [notify cookie 94669243886976 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.951+0000 7fc25d2d2700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e693dc00 con 0x5619e693d400 2024-09-13T23:43:33.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.951+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(66 notify.2 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.951+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94669243886976 notify 141733920795 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5619e6961380 con 0x5619e693d400 2024-09-13T23:43:33.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.951+0000 7fc264324f40 10 removing email index: 2024-09-13T23:43:33.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.951+0000 7fc264324f40 10 removing user buckets index 2024-09-13T23:43:33.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.951+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6b800 con 0x5619e693d000 2024-09-13T23:43:33.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.955+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(67 testx$tenanteduser.buckets [delete] v33'3 uv3 ondisk = 0) v8 ==== 170+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.955+0000 7fc264324f40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser.buckets, ofs0, ns] 2024-09-13T23:43:33.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.955+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [notify cookie 94669243886976 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6ac00 con 0x5619e693d400 2024-09-13T23:43:33.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.955+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== watch-notify(notify (1) cookie 94669242232064 notify 141733920796 ret 0) v3 ==== 217+0+0 (crc 0 0 0) 0x5619e6961380 con 0x5619e693d400 2024-09-13T23:43:33.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.955+0000 7fc25d2d2700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920796 cookie 94669242232064 notifier 4606 bl.length()=175 2024-09-13T23:43:33.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.955+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(68 notify.4 [notify cookie 94669243886976 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.955+0000 7fc25d2d2700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6c400 con 0x5619e693d400 2024-09-13T23:43:33.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.955+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(69 notify.4 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.955+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== watch-notify(notify_complete (2) cookie 94669243886976 notify 141733920796 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5619e6961520 con 0x5619e693d400 2024-09-13T23:43:33.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.955+0000 7fc264324f40 10 removing user index: testx$tenanteduser 2024-09-13T23:43:33.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.955+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6d400 con 0x5619e6a6a000 2024-09-13T23:43:33.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.959+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(70 testx$tenanteduser [call,call,delete] v33'8 uv8 ondisk = 0) v8 ==== 246+0+0 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.959+0000 7fc264324f40 10 removing default.rgw.meta+users.uid+testx$tenanteduser from cache 2024-09-13T23:43:33.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.959+0000 7fc264324f40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-09-13T23:43:33.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.959+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [notify cookie 94669243886976 in=179b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6d800 con 0x5619e6a6a400 2024-09-13T23:43:33.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.959+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== watch-notify(notify (1) cookie 94669242230912 notify 141733920788 ret 0) v3 ==== 209+0+0 (crc 0 0 0) 0x5619e6a11380 con 0x5619e6a6a400 2024-09-13T23:43:33.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.959+0000 7fc25d2d2700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920788 cookie 94669242230912 notifier 4606 bl.length()=167 2024-09-13T23:43:33.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.959+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(71 notify.3 [notify cookie 94669243886976 out=8b] v0'0 uv141 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.959+0000 7fc25d2d2700 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6bbc000 con 0x5619e6a6a400 2024-09-13T23:43:33.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.959+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== watch-notify(notify_complete (2) cookie 94669243886976 notify 141733920788 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5619e6a11520 con 0x5619e6a6a400 2024-09-13T23:43:33.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.959+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(72 notify.3 [notify-ack] v0'0 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.959+0000 7fc249aab700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:33.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.959+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94669243883520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6a6c400 con 0x5619e693d000 2024-09-13T23:43:33.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.963+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94669243883520] v33'138 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.963+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94669242227456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03400 con 0x5619e693d400 2024-09-13T23:43:33.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.967+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94669242227456] v33'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.967+0000 7fc264324f40 20 remove_watcher() i=0 2024-09-13T23:43:33.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.967+0000 7fc264324f40 2 removed watcher, disabling cache 2024-09-13T23:43:33.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.967+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94669242228608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03c00 con 0x5619e693d000 2024-09-13T23:43:33.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.967+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94669242228608] v33'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e693d000 2024-09-13T23:43:33.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.967+0000 7fc264324f40 20 remove_watcher() i=1 2024-09-13T23:43:33.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.967+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94669242229760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6b03800 con 0x5619e693d400 2024-09-13T23:43:33.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.971+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94669242229760] v33'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.971+0000 7fc264324f40 20 remove_watcher() i=2 2024-09-13T23:43:33.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.971+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94669242230912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6bbc000 con 0x5619e6a6a400 2024-09-13T23:43:33.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.971+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94669242230912] v33'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.971+0000 7fc264324f40 20 remove_watcher() i=3 2024-09-13T23:43:33.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.971+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94669242232064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6bbcc00 con 0x5619e693d400 2024-09-13T23:43:33.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.975+0000 7fc262add700 1 -- 172.21.15.167:0/796547858 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94669242232064] v33'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19200 con 0x5619e693d400 2024-09-13T23:43:33.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.975+0000 7fc264324f40 20 remove_watcher() i=4 2024-09-13T23:43:33.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.975+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94669242233216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6bbd000 con 0x5619e6a6a000 2024-09-13T23:43:33.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.979+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94669242233216] v33'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.979+0000 7fc264324f40 20 remove_watcher() i=5 2024-09-13T23:43:33.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.979+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94669242234368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6bbd400 con 0x5619e6a6a000 2024-09-13T23:43:33.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.979+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94669242234368] v33'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a19440 con 0x5619e6a6a000 2024-09-13T23:43:33.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.979+0000 7fc264324f40 20 remove_watcher() i=6 2024-09-13T23:43:33.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.979+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94669242235520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619e6bbd800 con 0x5619e6a6a400 2024-09-13T23:43:33.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc2632de700 1 -- 172.21.15.167:0/796547858 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94669242235520] v33'143 uv141 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619e6a18b40 con 0x5619e6a6a400 2024-09-13T23:43:33.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 20 remove_watcher() i=7 2024-09-13T23:43:33.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619e693d400 msgr2=0x5619e68e8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619e693d400 0x5619e68e8100 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619e693d000 msgr2=0x5619e68e7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619e693d000 0x5619e68e7b80 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619e6a6a400 msgr2=0x5619e68e8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619e6a6a400 0x5619e68e8c00 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619e6a6a000 msgr2=0x5619e68e8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619e6a6a000 0x5619e68e8680 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e693cc00 msgr2=0x5619e68e7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e693cc00 0x5619e68e7600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x5619e69abe30 tx=0x5619e6963f80 comp rx=0 tx=0).stop 2024-09-13T23:43:33.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 msgr2=0x5619e68e6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/796547858 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601800 0x5619e68e6b00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x5619e7461f50 tx=0x5619e6705da0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 reap_dead start 2024-09-13T23:43:33.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc2622dc700 1 -- 172.21.15.167:0/796547858 reap_dead start 2024-09-13T23:43:33.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 shutdown_connections 2024-09-13T23:43:33.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 >> 172.21.15.167:0/796547858 conn(0x5619e6601400 msgr2=0x5619e6701800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 shutdown_connections 2024-09-13T23:43:33.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/796547858 wait complete. 2024-09-13T23:43:33.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619e6a6d000 msgr2=0x5619e68e9700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619e6a6d000 0x5619e68e9700 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619e693d800 msgr2=0x5619e6b09080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619e693d800 0x5619e6b09080 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619e6a6c000 msgr2=0x5619e6b08580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619e6a6c000 0x5619e6b08580 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619e6a6a800 msgr2=0x5619e6b08b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619e6a6a800 0x5619e6b08b00 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e6b02000 msgr2=0x5619e6b08000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e6b02000 0x5619e6b08000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x5619e74751d0 tx=0x5619e69633e0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc262add700 1 -- 172.21.15.167:0/2581674119 reap_dead start 2024-09-13T23:43:33.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 msgr2=0x5619e68e9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 0x5619e68e9180 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x5619e6a7d350 tx=0x5619e6a7e2a0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 shutdown_connections 2024-09-13T23:43:33.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 --2- 172.21.15.167:0/2581674119 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6a6cc00 0x5619e68e9180 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 >> 172.21.15.167:0/2581674119 conn(0x5619e6a6dc00 msgr2=0x5619e6702000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 shutdown_connections 2024-09-13T23:43:33.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.983+0000 7fc264324f40 1 -- 172.21.15.167:0/2581674119 wait complete. 2024-09-13T23:43:33.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.987+0000 7fc264324f40 1 -- 172.21.15.167:0/3721836200 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e693c000 msgr2=0x5619e68e7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.987+0000 7fc264324f40 1 --2- 172.21.15.167:0/3721836200 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e693c000 0x5619e68e7080 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x5619e7475050 tx=0x5619e6962120 comp rx=0 tx=0).stop 2024-09-13T23:43:33.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.987+0000 7fc264324f40 1 -- 172.21.15.167:0/3721836200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 msgr2=0x5619e68e6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:33.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.987+0000 7fc264324f40 1 --2- 172.21.15.167:0/3721836200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5619e7461ef0 tx=0x5619e6705920 comp rx=0 tx=0).stop 2024-09-13T23:43:33.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.987+0000 7fc264324f40 1 -- 172.21.15.167:0/3721836200 shutdown_connections 2024-09-13T23:43:33.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.987+0000 7fc264324f40 1 --2- 172.21.15.167:0/3721836200 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619e693c000 0x5619e68e7080 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.987+0000 7fc264324f40 1 --2- 172.21.15.167:0/3721836200 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619e6601000 0x5619e68e6580 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:33.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.987+0000 7fc264324f40 1 -- 172.21.15.167:0/3721836200 >> 172.21.15.167:0/3721836200 conn(0x5619e6600c00 msgr2=0x5619e6701000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:33.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.987+0000 7fc264324f40 1 -- 172.21.15.167:0/3721836200 shutdown_connections 2024-09-13T23:43:33.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:33.987+0000 7fc264324f40 1 -- 172.21.15.167:0/3721836200 wait complete. 2024-09-13T23:43:34.014 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-09-13T23:43:34.019 INFO:tasks.radosgw_admin:RL: get_obj myfoo foo bytes_out=0 bytes_in=4 failed=False 2024-09-13T23:43:34.065 INFO:tasks.radosgw_admin:RL: list_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:34.066 INFO:tasks.util.rgw:rgwadmin: client.0 : ['object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-09-13T23:43:34.066 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-09-13T23:43:34.066 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph object rm --bucket myfoo --object four 2024-09-13T23:43:34.109 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:34.109 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:34.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.099+0000 7f39b3c22f40 1 Processor -- start 2024-09-13T23:43:34.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.099+0000 7f39b3c22f40 1 -- start start 2024-09-13T23:43:34.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.099+0000 7f39b3c22f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x559392a75400 0x559392d5a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.099+0000 7f39b3c22f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x559392b39500 con 0x559392a75000 2024-09-13T23:43:34.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.099+0000 7f39b3c22f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x559392b02a00 con 0x559392a75400 2024-09-13T23:43:34.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b1bda700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x559392a75400 0x559392d5a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b1bda700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x559392a75400 0x559392d5a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52858/0 (socket says 172.21.15.167:52858) 2024-09-13T23:43:34.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b23db700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x559392a75000 0x559392b74800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45558/0 (socket says 172.21.15.167:45558) 2024-09-13T23:43:34.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b1bda700 1 -- 172.21.15.167:0/2541930988 learned_addr learned my addr 172.21.15.167:0/2541930988 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b1bda700 1 -- 172.21.15.167:0/2541930988 >> v1:172.21.15.12:6789/0 conn(0x559392a75000 legacy=0x559392b74800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b1bda700 1 -- 172.21.15.167:0/2541930988 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559392a87d40 con 0x559392a75400 2024-09-13T23:43:34.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b1bda700 1 --2- 172.21.15.167:0/2541930988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75400 0x559392d5a580 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x559392d55920 tx=0x559392b79920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97e6c6c900a4c72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b13d9700 1 -- 172.21.15.167:0/2541930988 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392b194a0 con 0x559392a75400 2024-09-13T23:43:34.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b13d9700 1 -- 172.21.15.167:0/2541930988 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559392b39800 con 0x559392a75400 2024-09-13T23:43:34.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b13d9700 1 -- 172.21.15.167:0/2541930988 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392b19600 con 0x559392a75400 2024-09-13T23:43:34.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2541930988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75400 msgr2=0x559392d5a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2541930988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75400 0x559392d5a580 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x559392d55920 tx=0x559392b79920 comp rx=0 tx=0).stop 2024-09-13T23:43:34.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2541930988 shutdown_connections 2024-09-13T23:43:34.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2541930988 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75400 0x559392d5a580 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2541930988 >> 172.21.15.167:0/2541930988 conn(0x559392a74c00 msgr2=0x559392b74000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2541930988 shutdown_connections 2024-09-13T23:43:34.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2541930988 wait complete. 2024-09-13T23:43:34.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b3c22f40 1 Processor -- start 2024-09-13T23:43:34.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b3c22f40 1 -- start start 2024-09-13T23:43:34.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b3c22f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 0x559392d5a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.103+0000 7f39b3c22f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559392b02500 con 0x559392a75000 2024-09-13T23:43:34.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b23db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 0x559392d5a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b23db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 0x559392d5a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52862/0 (socket says 172.21.15.167:52862) 2024-09-13T23:43:34.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b23db700 1 -- 172.21.15.167:0/4071718956 learned_addr learned my addr 172.21.15.167:0/4071718956 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b23db700 1 -- 172.21.15.167:0/4071718956 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559392a87ba0 con 0x559392a75000 2024-09-13T23:43:34.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b23db700 1 --2- 172.21.15.167:0/4071718956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 0x559392d5a580 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x559392bd1e90 tx=0x559392b79b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217441512907fa07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b0bd8700 1 -- 172.21.15.167:0/4071718956 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392b194a0 con 0x559392a75000 2024-09-13T23:43:34.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b0bd8700 1 -- 172.21.15.167:0/4071718956 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559392b39b00 con 0x559392a75000 2024-09-13T23:43:34.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b0bd8700 1 -- 172.21.15.167:0/4071718956 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392b19600 con 0x559392a75000 2024-09-13T23:43:34.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b3c22f40 1 -- 172.21.15.167:0/4071718956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 msgr2=0x559392d5a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/4071718956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 0x559392d5a580 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x559392bd1e90 tx=0x559392b79b60 comp rx=0 tx=0).stop 2024-09-13T23:43:34.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b3c22f40 1 -- 172.21.15.167:0/4071718956 shutdown_connections 2024-09-13T23:43:34.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/4071718956 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 0x559392d5a580 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b3c22f40 1 -- 172.21.15.167:0/4071718956 >> 172.21.15.167:0/4071718956 conn(0x559392a74c00 msgr2=0x559392b75000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b3c22f40 1 -- 172.21.15.167:0/4071718956 shutdown_connections 2024-09-13T23:43:34.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b3c22f40 1 -- 172.21.15.167:0/4071718956 wait complete. 2024-09-13T23:43:34.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b3c22f40 1 Processor -- start 2024-09-13T23:43:34.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b3c22f40 1 -- start start 2024-09-13T23:43:34.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b3c22f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 0x559392d5a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b3c22f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559392b023c0 con 0x559392a75000 2024-09-13T23:43:34.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b23db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 0x559392d5a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b23db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 0x559392d5a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52876/0 (socket says 172.21.15.167:52876) 2024-09-13T23:43:34.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.107+0000 7f39b23db700 1 -- 172.21.15.167:0/2795861554 learned_addr learned my addr 172.21.15.167:0/2795861554 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b23db700 1 -- 172.21.15.167:0/2795861554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559392a87a00 con 0x559392a75000 2024-09-13T23:43:34.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b23db700 1 --2- 172.21.15.167:0/2795861554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 0x559392d5a580 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x5593938da0c0 tx=0x559392b79a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39af3d5700 1 -- 172.21.15.167:0/2795861554 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392b198c0 con 0x559392a75000 2024-09-13T23:43:34.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2795861554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559392a87860 con 0x559392a75000 2024-09-13T23:43:34.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39af3d5700 1 -- 172.21.15.167:0/2795861554 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559392b39c80 con 0x559392a75000 2024-09-13T23:43:34.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39af3d5700 1 -- 172.21.15.167:0/2795861554 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392b194a0 con 0x559392a75000 2024-09-13T23:43:34.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2795861554 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559392a876c0 con 0x559392a75000 2024-09-13T23:43:34.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39af3d5700 1 -- 172.21.15.167:0/2795861554 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x559392b6eb00 con 0x559392a75000 2024-09-13T23:43:34.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39af3d5700 1 --2- 172.21.15.167:0/2795861554 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392e0c000 0x559392d5b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39af3d5700 1 -- 172.21.15.167:0/2795861554 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x559392adb180 con 0x559392a75000 2024-09-13T23:43:34.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b3c22f40 1 Processor -- start 2024-09-13T23:43:34.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b1bda700 1 --2- 172.21.15.167:0/2795861554 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392e0c000 0x559392d5b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b3c22f40 1 -- start start 2024-09-13T23:43:34.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b3c22f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 0x559392d5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b3c22f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559392a9a640 con 0x559392a75800 2024-09-13T23:43:34.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b1bda700 1 --2- 172.21.15.167:0/2795861554 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392e0c000 0x559392d5b080 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x559392d55920 tx=0x559392b79800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b23db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 0x559392d5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b23db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 0x559392d5ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52882/0 (socket says 172.21.15.167:52882) 2024-09-13T23:43:34.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.111+0000 7f39b23db700 1 -- 172.21.15.167:0/2253432930 learned_addr learned my addr 172.21.15.167:0/2253432930 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b23db700 1 -- 172.21.15.167:0/2253432930 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559392a86000 con 0x559392a75800 2024-09-13T23:43:34.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b23db700 1 --2- 172.21.15.167:0/2253432930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 0x559392d5ab00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x5593938db0b0 tx=0x559392dc8240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfad163f9a87b59 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39acbd0700 1 -- 172.21.15.167:0/2253432930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392b19600 con 0x559392a75800 2024-09-13T23:43:34.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39acbd0700 1 -- 172.21.15.167:0/2253432930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559392b39c80 con 0x559392a75800 2024-09-13T23:43:34.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39acbd0700 1 -- 172.21.15.167:0/2253432930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392b19a20 con 0x559392a75800 2024-09-13T23:43:34.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2253432930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 msgr2=0x559392d5ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2253432930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 0x559392d5ab00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x5593938db0b0 tx=0x559392dc8240 comp rx=0 tx=0).stop 2024-09-13T23:43:34.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2253432930 shutdown_connections 2024-09-13T23:43:34.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2253432930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 0x559392d5ab00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2253432930 >> 172.21.15.167:0/2253432930 conn(0x559392a75400 msgr2=0x559392b75800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2253432930 shutdown_connections 2024-09-13T23:43:34.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2253432930 wait complete. 2024-09-13T23:43:34.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b3c22f40 1 Processor -- start 2024-09-13T23:43:34.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b3c22f40 1 -- start start 2024-09-13T23:43:34.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b3c22f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 0x559392d5ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b3c22f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559392a9a8c0 con 0x559392a75800 2024-09-13T23:43:34.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b23db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 0x559392d5ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b23db700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 0x559392d5ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52884/0 (socket says 172.21.15.167:52884) 2024-09-13T23:43:34.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.115+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 learned_addr learned my addr 172.21.15.167:0/1778783266 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559392a86ea0 con 0x559392a75800 2024-09-13T23:43:34.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39b23db700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 0x559392d5ab00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x559392e08150 tx=0x559392dc8d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39ab3cd700 1 -- 172.21.15.167:0/1778783266 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392b19600 con 0x559392a75800 2024-09-13T23:43:34.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559392a86820 con 0x559392a75800 2024-09-13T23:43:34.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39ab3cd700 1 -- 172.21.15.167:0/1778783266 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559392e32000 con 0x559392a75800 2024-09-13T23:43:34.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39ab3cd700 1 -- 172.21.15.167:0/1778783266 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392b19a20 con 0x559392a75800 2024-09-13T23:43:34.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559392a869c0 con 0x559392a75800 2024-09-13T23:43:34.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39ab3cd700 1 -- 172.21.15.167:0/1778783266 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x559392b6edc0 con 0x559392a75800 2024-09-13T23:43:34.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39ab3cd700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392e0d000 0x559392d5b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39ab3cd700 1 -- 172.21.15.167:0/1778783266 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x559392adb340 con 0x559392a75800 2024-09-13T23:43:34.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39b1bda700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392e0d000 0x559392d5b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559392e0d400 0x559392d5bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.119+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e0d800 con 0x559392e0d400 2024-09-13T23:43:34.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b2bdc700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559392e0d400 0x559392d5bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b1bda700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392e0d000 0x559392d5b600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x559392d54c30 tx=0x559392b79860 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b2bdc700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559392e0d400 0x559392d5bb80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 20 realm 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e0dc00 con 0x559392e0d400 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e0d800 con 0x559392e0d400 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559392e0dc00 0x559392d5c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa000 con 0x559392e0dc00 2024-09-13T23:43:34.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b23db700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559392e0dc00 0x559392d5c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b23db700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559392e0dc00 0x559392d5c100 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:34.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.123+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0dc00 2024-09-13T23:43:34.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b3c22f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:34.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b3c22f40 20 searching for the correct realm 2024-09-13T23:43:34.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559392eaa800 0x559392d5c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaac00 con 0x559392eaa800 2024-09-13T23:43:34.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b1bda700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559392eaa800 0x559392d5c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b1bda700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559392eaa800 0x559392d5c680 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaac00 con 0x559392e0dc00 2024-09-13T23:43:34.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b23db700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559392eaa000 0x559392d5cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa000 2024-09-13T23:43:34.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b2bdc700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559392eaa000 0x559392d5cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b2bdc700 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559392eaa000 0x559392d5cc00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.127+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0d400 2024-09-13T23:43:34.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0d400 2024-09-13T23:43:34.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa800 2024-09-13T23:43:34.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0d400 2024-09-13T23:43:34.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0dc00 2024-09-13T23:43:34.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0dc00 2024-09-13T23:43:34.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0d400 2024-09-13T23:43:34.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.131+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa000 2024-09-13T23:43:34.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0d400 2024-09-13T23:43:34.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa000 2024-09-13T23:43:34.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0dc00 2024-09-13T23:43:34.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa800 2024-09-13T23:43:34.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0d400 2024-09-13T23:43:34.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0dc00 2024-09-13T23:43:34.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.135+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa800 2024-09-13T23:43:34.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0d400 2024-09-13T23:43:34.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0d400 2024-09-13T23:43:34.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0d400 2024-09-13T23:43:34.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa800 2024-09-13T23:43:34.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa000 2024-09-13T23:43:34.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0d400 2024-09-13T23:43:34.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.139+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa000 2024-09-13T23:43:34.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa800 2024-09-13T23:43:34.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa000 2024-09-13T23:43:34.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0dc00 2024-09-13T23:43:34.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0dc00 2024-09-13T23:43:34.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa800 2024-09-13T23:43:34.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0dc00 2024-09-13T23:43:34.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.143+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0dc00 2024-09-13T23:43:34.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:34.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:34.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:34.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:34.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab000 con 0x559392e0d400 2024-09-13T23:43:34.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab400 con 0x559392e0dc00 2024-09-13T23:43:34.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:34.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab800 con 0x559392e0dc00 2024-09-13T23:43:34.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:34.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 zone default found 2024-09-13T23:43:34.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 4 Realm: () 2024-09-13T23:43:34.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:34.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:34.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:34.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 zonegroup default 2024-09-13T23:43:34.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eabc00 con 0x559392eaa000 2024-09-13T23:43:34.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.147+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab000 con 0x559392eaa800 2024-09-13T23:43:34.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.151+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.151+0000 7f39b3c22f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.151+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.151+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eabc00 con 0x559392eaa800 2024-09-13T23:43:34.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.151+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.151+0000 7f39b3c22f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.151+0000 7f39b3c22f40 20 started sync module instance, tier type = 2024-09-13T23:43:34.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.151+0000 7f39b3c22f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:34.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.151+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e0d800 con 0x559392e0dc00 2024-09-13T23:43:34.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.151+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.151+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94092311993088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaac00 con 0x559392e0dc00 2024-09-13T23:43:34.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.151+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392e0d400 2024-09-13T23:43:34.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.155+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.155+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94092311993088] v33'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.155+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94092311994240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab400 con 0x559392e0d400 2024-09-13T23:43:34.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.155+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab800 con 0x559392e0dc00 2024-09-13T23:43:34.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.159+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94092311994240] v33'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.159+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.159+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94092311995392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab000 con 0x559392e0dc00 2024-09-13T23:43:34.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.159+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eabc00 con 0x559392eaa000 2024-09-13T23:43:34.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.163+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.163+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94092311996544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e0c800 con 0x559392eaa000 2024-09-13T23:43:34.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.163+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e0d800 con 0x559392e0dc00 2024-09-13T23:43:34.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.163+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94092311995392] v33'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.163+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94092311996544] v33'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.163+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.163+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94092311997696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e0c800 con 0x559392e0dc00 2024-09-13T23:43:34.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.163+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa800 2024-09-13T23:43:34.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.167+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.167+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94092311997696] v33'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.167+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94092311998848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab400 con 0x559392eaa800 2024-09-13T23:43:34.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.167+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eabc00 con 0x559392eaa800 2024-09-13T23:43:34.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.167+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94092311998848] v33'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.167+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.167+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94092312000000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaa400 con 0x559392eaa800 2024-09-13T23:43:34.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.167+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab400 con 0x559392eaa000 2024-09-13T23:43:34.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94092312000000] v33'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'144 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94092312001152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eabc00 con 0x559392eaa000 2024-09-13T23:43:34.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b3c22f40 20 add_watcher() i=0 2024-09-13T23:43:34.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b3c22f40 20 add_watcher() i=1 2024-09-13T23:43:34.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b3c22f40 20 add_watcher() i=2 2024-09-13T23:43:34.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b3c22f40 20 add_watcher() i=3 2024-09-13T23:43:34.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b3c22f40 20 add_watcher() i=4 2024-09-13T23:43:34.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b3c22f40 20 add_watcher() i=5 2024-09-13T23:43:34.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b3c22f40 20 add_watcher() i=6 2024-09-13T23:43:34.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94092312001152] v33'145 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b3c22f40 20 add_watcher() i=7 2024-09-13T23:43:34.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b3c22f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:34.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.171+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eba000 con 0x559392e0d400 2024-09-13T23:43:34.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94092313427968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eba400 con 0x559392e0d400 2024-09-13T23:43:34.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94092313427968] v33'139 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39993a9700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:34.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39b3c22f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:34.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39b3c22f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:34.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39b3c22f40 1 Processor -- start 2024-09-13T23:43:34.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39b3c22f40 1 -- start start 2024-09-13T23:43:34.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39b3c22f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 0x559392d5d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39b3c22f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559392b023c0 con 0x559392ebac00 2024-09-13T23:43:34.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39b1bda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 0x559392d5d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39b1bda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 0x559392d5d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52898/0 (socket says 172.21.15.167:52898) 2024-09-13T23:43:34.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.175+0000 7f39b1bda700 1 -- 172.21.15.167:0/2716588877 learned_addr learned my addr 172.21.15.167:0/2716588877 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b1bda700 1 -- 172.21.15.167:0/2716588877 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559392a86b60 con 0x559392ebac00 2024-09-13T23:43:34.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b1bda700 1 --2- 172.21.15.167:0/2716588877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 0x559392d5d180 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5593938db1a0 tx=0x559392e94180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f765ad0a0af3f2cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f3998ba8700 1 -- 172.21.15.167:0/2716588877 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392b19760 con 0x559392ebac00 2024-09-13T23:43:34.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f3998ba8700 1 -- 172.21.15.167:0/2716588877 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559392e32900 con 0x559392ebac00 2024-09-13T23:43:34.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f3998ba8700 1 -- 172.21.15.167:0/2716588877 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392ebc160 con 0x559392ebac00 2024-09-13T23:43:34.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2716588877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 msgr2=0x559392d5d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2716588877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 0x559392d5d180 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5593938db1a0 tx=0x559392e94180 comp rx=0 tx=0).stop 2024-09-13T23:43:34.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2716588877 shutdown_connections 2024-09-13T23:43:34.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2716588877 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 0x559392d5d180 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2716588877 >> 172.21.15.167:0/2716588877 conn(0x559392eba800 msgr2=0x559392b76000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2716588877 shutdown_connections 2024-09-13T23:43:34.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2716588877 wait complete. 2024-09-13T23:43:34.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b3c22f40 1 Processor -- start 2024-09-13T23:43:34.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b3c22f40 1 -- start start 2024-09-13T23:43:34.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b3c22f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 0x559392d5d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b3c22f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x559392b02500 con 0x559392ebac00 2024-09-13T23:43:34.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b1bda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 0x559392d5d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b1bda700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 0x559392d5d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52914/0 (socket says 172.21.15.167:52914) 2024-09-13T23:43:34.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.179+0000 7f39b1bda700 1 -- 172.21.15.167:0/2980258984 learned_addr learned my addr 172.21.15.167:0/2980258984 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39b1bda700 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559392a86d00 con 0x559392ebac00 2024-09-13T23:43:34.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39b1bda700 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 0x559392d5d180 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x559392ec9260 tx=0x559392e940c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39973a5700 1 -- 172.21.15.167:0/2980258984 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392ebc2c0 con 0x559392ebac00 2024-09-13T23:43:34.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39973a5700 1 -- 172.21.15.167:0/2980258984 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559392e32600 con 0x559392ebac00 2024-09-13T23:43:34.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559392a86340 con 0x559392ebac00 2024-09-13T23:43:34.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39973a5700 1 -- 172.21.15.167:0/2980258984 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x559392ebc420 con 0x559392ebac00 2024-09-13T23:43:34.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559392a87d40 con 0x559392ebac00 2024-09-13T23:43:34.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39973a5700 1 -- 172.21.15.167:0/2980258984 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x559392b6fb80 con 0x559392ebac00 2024-09-13T23:43:34.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39973a5700 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392f42000 0x559392f48000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39973a5700 1 -- 172.21.15.167:0/2980258984 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x559392adb6c0 con 0x559392ebac00 2024-09-13T23:43:34.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559392ebb000 0x559392d5d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39b2bdc700 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392f42000 0x559392f48000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392ebb400 con 0x559392ebb000 2024-09-13T23:43:34.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39b23db700 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559392ebb000 0x559392d5d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39b2bdc700 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392f42000 0x559392f48000 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x559392e455f0 tx=0x559392e94360 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.183+0000 7f39b23db700 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559392ebb000 0x559392d5d700 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.187+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v33'29 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.187+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559392ebb800 0x559392f48580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.187+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392ebbc00 con 0x559392ebb800 2024-09-13T23:43:34.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.187+0000 7f39b1bda700 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559392ebb800 0x559392f48580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.187+0000 7f39b1bda700 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559392ebb800 0x559392f48580 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.191+0000 7f39b1bda700 1 -- 172.21.15.167:0/2980258984 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v33'17 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559392ea5440 con 0x559392ebb800 2024-09-13T23:43:34.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.191+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559392f42400 0x559392f48b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.191+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f42800 con 0x559392f42400 2024-09-13T23:43:34.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.191+0000 7f39b2bdc700 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559392f42400 0x559392f48b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.191+0000 7f39b2bdc700 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559392f42400 0x559392f48b00 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.195+0000 7f39b2bdc700 1 -- 172.21.15.167:0/2980258984 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v33'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392f42400 2024-09-13T23:43:34.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.195+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559392f42c00 0x559392f49080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.195+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f43000 con 0x559392f42c00 2024-09-13T23:43:34.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.195+0000 7f39b23db700 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559392f42c00 0x559392f49080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.195+0000 7f39b23db700 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559392f42c00 0x559392f49080 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.195+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v33'140 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392f42c00 2024-09-13T23:43:34.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.195+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f43400 con 0x559392ebb000 2024-09-13T23:43:34.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.199+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v33'41 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.199+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f43800 con 0x559392f42400 2024-09-13T23:43:34.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.203+0000 7f39b2bdc700 1 -- 172.21.15.167:0/2980258984 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v33'30 uv10 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392f42400 2024-09-13T23:43:34.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.203+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f43c00 con 0x559392ebb000 2024-09-13T23:43:34.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.203+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v33'19 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.203+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f43000 con 0x559392f42c00 2024-09-13T23:43:34.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.207+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v33'28 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392f42c00 2024-09-13T23:43:34.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.207+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f42800 con 0x559392ebb000 2024-09-13T23:43:34.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.211+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v33'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.211+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392ebb400 con 0x559392ebb000 2024-09-13T23:43:34.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.211+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v33'24 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.211+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eba000 con 0x559392ebb000 2024-09-13T23:43:34.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.215+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v33'42 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.215+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eba400 con 0x559392f42400 2024-09-13T23:43:34.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.219+0000 7f39b2bdc700 1 -- 172.21.15.167:0/2980258984 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v33'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392f42400 2024-09-13T23:43:34.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.219+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaac00 con 0x559392ebb000 2024-09-13T23:43:34.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.219+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v33'17 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.223+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab800 con 0x559392f42c00 2024-09-13T23:43:34.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.223+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v33'33 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392f42c00 2024-09-13T23:43:34.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.223+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab000 con 0x559392f42c00 2024-09-13T23:43:34.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.227+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v33'141 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392f42c00 2024-09-13T23:43:34.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.227+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab400 con 0x559392ebb000 2024-09-13T23:43:34.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.231+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v33'35 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.231+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eabc00 con 0x559392ebb000 2024-09-13T23:43:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.231+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v33'43 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.231+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e0d800 con 0x559392f42400 2024-09-13T23:43:34.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.235+0000 7f39b2bdc700 1 -- 172.21.15.167:0/2980258984 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v33'17 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392f42400 2024-09-13T23:43:34.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.235+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e0c800 con 0x559392ebb000 2024-09-13T23:43:34.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.239+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v33'25 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.239+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab800 con 0x559392f42c00 2024-09-13T23:43:34.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.243+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v33'142 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392f42c00 2024-09-13T23:43:34.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.243+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab000 con 0x559392ebb000 2024-09-13T23:43:34.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.243+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v33'36 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.243+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab400 con 0x559392ebb000 2024-09-13T23:43:34.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.247+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v33'27 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.247+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eaac00 con 0x559392f42400 2024-09-13T23:43:34.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.251+0000 7f39b2bdc700 1 -- 172.21.15.167:0/2980258984 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v33'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392f42400 2024-09-13T23:43:34.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.251+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392ebb400 con 0x559392ebb000 2024-09-13T23:43:34.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.251+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v33'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.251+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eba000 con 0x559392f42400 2024-09-13T23:43:34.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.255+0000 7f39b2bdc700 1 -- 172.21.15.167:0/2980258984 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v33'13 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392f42400 2024-09-13T23:43:34.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.255+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f43400 con 0x559392f42400 2024-09-13T23:43:34.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.259+0000 7f39b2bdc700 1 -- 172.21.15.167:0/2980258984 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v33'31 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392f42400 2024-09-13T23:43:34.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.259+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f43c00 con 0x559392f42400 2024-09-13T23:43:34.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.263+0000 7f39b2bdc700 1 -- 172.21.15.167:0/2980258984 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v33'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392f42400 2024-09-13T23:43:34.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.263+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f43000 con 0x559392ebb000 2024-09-13T23:43:34.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.263+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v33'17 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.263+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f42800 con 0x559392f42c00 2024-09-13T23:43:34.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.267+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v33'29 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392f42c00 2024-09-13T23:43:34.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.267+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f43800 con 0x559392ebb000 2024-09-13T23:43:34.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.271+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v33'37 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392ebb000 2024-09-13T23:43:34.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.271+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f43400 con 0x559392f42c00 2024-09-13T23:43:34.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.271+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v33'34 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392f42c00 2024-09-13T23:43:34.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.271+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eba400 con 0x559392ebb800 2024-09-13T23:43:34.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.275+0000 7f39b1bda700 1 -- 172.21.15.167:0/2980258984 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v33'23 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x559392ea5440 con 0x559392ebb800 2024-09-13T23:43:34.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.275+0000 7f3994ba0700 20 reqs_thread_entry: start 2024-09-13T23:43:34.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.275+0000 7f39b3c22f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:34.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:34.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f3992b9c700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:04 2024 2024-09-13T23:43:34.287 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:34.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f399237a700 20 reqs_thread_entry: start 2024-09-13T23:43:34.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-13T23:43:34.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e0d800 con 0x559392eaa000 2024-09-13T23:43:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-13T23:43:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 : miss 2024-09-13T23:43:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f43000 con 0x559392eaa800 2024-09-13T23:43:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:34.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:34.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:34.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:34.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:34.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 2024-09-13T23:43:34.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 20 get_obj_state: rctx=0x559392b05b20 obj=myfoo:four state=0x559392ff81e8 s->prefetch_data=0 2024-09-13T23:43:34.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 3.ds0 3:b3ff8916:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1_four:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f42800 con 0x559392e0dc00 2024-09-13T23:43:34.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1_four [getxattrs out=851b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 234+0+867 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 10 manifest: total_size = 4 2024-09-13T23:43:34.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 20 get_obj_state: setting s->obj_tag to 9144208c-3c85-4fb6-9007-2693d5a930f0.4222.17936564074699706503 2024-09-13T23:43:34.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 20 get_obj_state: rctx=0x559392b05b20 obj=myfoo:four state=0x559392ff81e8 s->prefetch_data=0 2024-09-13T23:43:34.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 20 prepare_atomic_modification: state is not atomic. state=0x559392ff81e8 2024-09-13T23:43:34.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 20 bucket index object: default.rgw.buckets.index:.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6 2024-09-13T23:43:34.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.279+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 4.2c 4:355fcb42:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=182b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392f43800 con 0x559392e0d400 2024-09-13T23:43:34.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.283+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(65 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6 [stat,call,call] v33'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.283+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 3.ds0 3:b3ff8916:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1_four:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392ebb400 con 0x559392e0dc00 2024-09-13T23:43:34.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.287+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1_four [call] v33'2 uv2 ondisk = 0) v8 ==== 192+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.287+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 4.2c 4:355fcb42:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=259b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab800 con 0x559392e0d400 2024-09-13T23:43:34.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.291+0000 7f39993a9700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:34.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.291+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94092313427968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab000 con 0x559392e0d400 2024-09-13T23:43:34.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.291+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(67 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6 [stat,call,call] v33'8 uv8 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.291+0000 7f39acbd0700 20 handle_completion(): completion ok for obj=four 2024-09-13T23:43:34.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.291+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(68 data_loggenerations_metadata [watch unwatch cookie 94092313427968] v33'143 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.291+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94092311993088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392eab400 con 0x559392e0dc00 2024-09-13T23:43:34.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.295+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.0 [watch unwatch cookie 94092311993088] v33'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.295+0000 7f39b3c22f40 20 remove_watcher() i=0 2024-09-13T23:43:34.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.295+0000 7f39b3c22f40 2 removed watcher, disabling cache 2024-09-13T23:43:34.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.295+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94092311994240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e0c800 con 0x559392e0d400 2024-09-13T23:43:34.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.299+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(70 notify.1 [watch unwatch cookie 94092311994240] v33'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392e0d400 2024-09-13T23:43:34.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.299+0000 7f39b3c22f40 20 remove_watcher() i=1 2024-09-13T23:43:34.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.299+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94092311995392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e90000 con 0x559392e0dc00 2024-09-13T23:43:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.299+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(71 notify.2 [watch unwatch cookie 94092311995392] v33'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.299+0000 7f39b3c22f40 20 remove_watcher() i=2 2024-09-13T23:43:34.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.299+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94092311996544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e90400 con 0x559392eaa000 2024-09-13T23:43:34.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.303+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(72 notify.3 [watch unwatch cookie 94092311996544] v33'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.303+0000 7f39b3c22f40 20 remove_watcher() i=3 2024-09-13T23:43:34.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.303+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94092311997696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e90800 con 0x559392e0dc00 2024-09-13T23:43:34.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.303+0000 7f39b23db700 1 -- 172.21.15.167:0/1778783266 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(73 notify.4 [watch unwatch cookie 94092311997696] v33'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5200 con 0x559392e0dc00 2024-09-13T23:43:34.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.303+0000 7f39b3c22f40 20 remove_watcher() i=4 2024-09-13T23:43:34.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.303+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:74 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94092311998848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e90c00 con 0x559392eaa800 2024-09-13T23:43:34.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.307+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(74 notify.5 [watch unwatch cookie 94092311998848] v33'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.307+0000 7f39b3c22f40 20 remove_watcher() i=5 2024-09-13T23:43:34.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.307+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:75 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94092312000000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e91000 con 0x559392eaa800 2024-09-13T23:43:34.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.307+0000 7f39b1bda700 1 -- 172.21.15.167:0/1778783266 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(75 notify.6 [watch unwatch cookie 94092312000000] v33'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea5440 con 0x559392eaa800 2024-09-13T23:43:34.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.307+0000 7f39b3c22f40 20 remove_watcher() i=6 2024-09-13T23:43:34.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.307+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94092312001152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x559392e91400 con 0x559392eaa000 2024-09-13T23:43:34.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(76 notify.7 [watch unwatch cookie 94092312001152] v33'146 uv144 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x559392ea4b40 con 0x559392eaa000 2024-09-13T23:43:34.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 20 remove_watcher() i=7 2024-09-13T23:43:34.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559392e0dc00 msgr2=0x559392d5c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559392e0dc00 0x559392d5c100 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559392e0d400 msgr2=0x559392d5bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559392e0d400 0x559392d5bb80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559392eaa000 msgr2=0x559392d5cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559392eaa000 0x559392d5cc00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559392eaa800 msgr2=0x559392d5c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559392eaa800 0x559392d5c680 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392e0d000 msgr2=0x559392d5b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392e0d000 0x559392d5b600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x559392d54c30 tx=0x559392b79860 comp rx=0 tx=0).stop 2024-09-13T23:43:34.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 msgr2=0x559392d5ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b2bdc700 1 -- 172.21.15.167:0/1778783266 reap_dead start 2024-09-13T23:43:34.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 0x559392d5ab00 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x559392e08150 tx=0x559392dc8d80 comp rx=0 tx=0).stop 2024-09-13T23:43:34.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 shutdown_connections 2024-09-13T23:43:34.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/1778783266 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75800 0x559392d5ab00 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 >> 172.21.15.167:0/1778783266 conn(0x559392a75400 msgr2=0x559392b75800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 shutdown_connections 2024-09-13T23:43:34.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/1778783266 wait complete. 2024-09-13T23:43:34.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559392ebb000 msgr2=0x559392d5d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x559392ebb000 0x559392d5d700 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559392f42c00 msgr2=0x559392f49080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x559392f42c00 0x559392f49080 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559392ebb800 msgr2=0x559392f48580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x559392ebb800 0x559392f48580 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559392f42400 msgr2=0x559392f48b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x559392f42400 0x559392f48b00 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392f42000 msgr2=0x559392f48000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392f42000 0x559392f48000 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x559392e455f0 tx=0x559392e94360 comp rx=0 tx=0).stop 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 msgr2=0x559392d5d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2980258984 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392ebac00 0x559392d5d180 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x559392ec9260 tx=0x559392e940c0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b23db700 1 -- 172.21.15.167:0/2980258984 reap_dead start 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 shutdown_connections 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 >> 172.21.15.167:0/2980258984 conn(0x559392eba800 msgr2=0x559392b76000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 shutdown_connections 2024-09-13T23:43:34.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.311+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2980258984 wait complete. 2024-09-13T23:43:34.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.315+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2795861554 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392e0c000 msgr2=0x559392d5b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.315+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2795861554 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392e0c000 0x559392d5b080 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x559392d55920 tx=0x559392b79800 comp rx=0 tx=0).stop 2024-09-13T23:43:34.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.315+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2795861554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 msgr2=0x559392d5a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.315+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2795861554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 0x559392d5a580 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x5593938da0c0 tx=0x559392b79a40 comp rx=0 tx=0).stop 2024-09-13T23:43:34.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.315+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2795861554 shutdown_connections 2024-09-13T23:43:34.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.315+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2795861554 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x559392e0c000 0x559392d5b080 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.315+0000 7f39b3c22f40 1 --2- 172.21.15.167:0/2795861554 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x559392a75000 0x559392d5a580 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.315+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2795861554 >> 172.21.15.167:0/2795861554 conn(0x559392a74c00 msgr2=0x559392b75000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.315+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2795861554 shutdown_connections 2024-09-13T23:43:34.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.315+0000 7f39b3c22f40 1 -- 172.21.15.167:0/2795861554 wait complete. 2024-09-13T23:43:34.329 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-09-13T23:43:34.329 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-09-13T23:43:34.329 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-09-13T23:43:34.367 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:34.367 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:34.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4d544f40 1 Processor -- start 2024-09-13T23:43:34.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4d544f40 1 -- start start 2024-09-13T23:43:34.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4d544f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ea2c83f400 0x55ea2cb24580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4d544f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ea2c903500 con 0x55ea2c83f000 2024-09-13T23:43:34.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4d544f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ea2c8cca00 con 0x55ea2c83f400 2024-09-13T23:43:34.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4b4fc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ea2c83f400 0x55ea2cb24580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4b4fc700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ea2c83f400 0x55ea2cb24580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52916/0 (socket says 172.21.15.167:52916) 2024-09-13T23:43:34.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4bcfd700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55ea2c83f000 0x55ea2c93e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45560/0 (socket says 172.21.15.167:45560) 2024-09-13T23:43:34.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4054948282 learned_addr learned my addr 172.21.15.167:0/4054948282 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4054948282 >> v1:172.21.15.12:6789/0 conn(0x55ea2c83f000 legacy=0x55ea2c93e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4054948282 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea2c851d40 con 0x55ea2c83f400 2024-09-13T23:43:34.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4b4fc700 1 --2- 172.21.15.167:0/4054948282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f400 0x55ea2cb24580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55ea2cb1f920 tx=0x55ea2c943920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f24c28150649fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4acfb700 1 -- 172.21.15.167:0/4054948282 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2c8e34a0 con 0x55ea2c83f400 2024-09-13T23:43:34.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4acfb700 1 -- 172.21.15.167:0/4054948282 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ea2c903800 con 0x55ea2c83f400 2024-09-13T23:43:34.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.359+0000 7fea4acfb700 1 -- 172.21.15.167:0/4054948282 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2c8e3600 con 0x55ea2c83f400 2024-09-13T23:43:34.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 -- 172.21.15.167:0/4054948282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f400 msgr2=0x55ea2cb24580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 --2- 172.21.15.167:0/4054948282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f400 0x55ea2cb24580 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55ea2cb1f920 tx=0x55ea2c943920 comp rx=0 tx=0).stop 2024-09-13T23:43:34.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 -- 172.21.15.167:0/4054948282 shutdown_connections 2024-09-13T23:43:34.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 --2- 172.21.15.167:0/4054948282 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f400 0x55ea2cb24580 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 -- 172.21.15.167:0/4054948282 >> 172.21.15.167:0/4054948282 conn(0x55ea2c83ec00 msgr2=0x55ea2c93e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 -- 172.21.15.167:0/4054948282 shutdown_connections 2024-09-13T23:43:34.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 -- 172.21.15.167:0/4054948282 wait complete. 2024-09-13T23:43:34.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 Processor -- start 2024-09-13T23:43:34.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 -- start start 2024-09-13T23:43:34.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 0x55ea2cb24580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c8cc500 con 0x55ea2c83f000 2024-09-13T23:43:34.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4bcfd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 0x55ea2cb24580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4bcfd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 0x55ea2cb24580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52928/0 (socket says 172.21.15.167:52928) 2024-09-13T23:43:34.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4bcfd700 1 -- 172.21.15.167:0/3898905174 learned_addr learned my addr 172.21.15.167:0/3898905174 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4bcfd700 1 -- 172.21.15.167:0/3898905174 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea2c851ba0 con 0x55ea2c83f000 2024-09-13T23:43:34.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4bcfd700 1 --2- 172.21.15.167:0/3898905174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 0x55ea2cb24580 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55ea2c99be90 tx=0x55ea2c943b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9099d3c803d8efae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4a4fa700 1 -- 172.21.15.167:0/3898905174 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2c8e34a0 con 0x55ea2c83f000 2024-09-13T23:43:34.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4a4fa700 1 -- 172.21.15.167:0/3898905174 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ea2c903b00 con 0x55ea2c83f000 2024-09-13T23:43:34.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4a4fa700 1 -- 172.21.15.167:0/3898905174 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2c8e3600 con 0x55ea2c83f000 2024-09-13T23:43:34.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 -- 172.21.15.167:0/3898905174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 msgr2=0x55ea2cb24580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 --2- 172.21.15.167:0/3898905174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 0x55ea2cb24580 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55ea2c99be90 tx=0x55ea2c943b60 comp rx=0 tx=0).stop 2024-09-13T23:43:34.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 -- 172.21.15.167:0/3898905174 shutdown_connections 2024-09-13T23:43:34.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 --2- 172.21.15.167:0/3898905174 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 0x55ea2cb24580 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 -- 172.21.15.167:0/3898905174 >> 172.21.15.167:0/3898905174 conn(0x55ea2c83ec00 msgr2=0x55ea2c93f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 -- 172.21.15.167:0/3898905174 shutdown_connections 2024-09-13T23:43:34.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.363+0000 7fea4d544f40 1 -- 172.21.15.167:0/3898905174 wait complete. 2024-09-13T23:43:34.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4d544f40 1 Processor -- start 2024-09-13T23:43:34.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4d544f40 1 -- start start 2024-09-13T23:43:34.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4d544f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 0x55ea2cb24580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4d544f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c8cc3c0 con 0x55ea2c83f000 2024-09-13T23:43:34.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4bcfd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 0x55ea2cb24580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4bcfd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 0x55ea2cb24580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52934/0 (socket says 172.21.15.167:52934) 2024-09-13T23:43:34.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4bcfd700 1 -- 172.21.15.167:0/1925205237 learned_addr learned my addr 172.21.15.167:0/1925205237 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4bcfd700 1 -- 172.21.15.167:0/1925205237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea2c851a00 con 0x55ea2c83f000 2024-09-13T23:43:34.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4bcfd700 1 --2- 172.21.15.167:0/1925205237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 0x55ea2cb24580 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55ea2c99bfb0 tx=0x55ea2c943a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea48cf7700 1 -- 172.21.15.167:0/1925205237 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2c8e38c0 con 0x55ea2c83f000 2024-09-13T23:43:34.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea48cf7700 1 -- 172.21.15.167:0/1925205237 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ea2c903c80 con 0x55ea2c83f000 2024-09-13T23:43:34.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4d544f40 1 -- 172.21.15.167:0/1925205237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ea2c851860 con 0x55ea2c83f000 2024-09-13T23:43:34.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea48cf7700 1 -- 172.21.15.167:0/1925205237 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2c8e34a0 con 0x55ea2c83f000 2024-09-13T23:43:34.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4d544f40 1 -- 172.21.15.167:0/1925205237 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea2c8516c0 con 0x55ea2c83f000 2024-09-13T23:43:34.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4d544f40 1 Processor -- start 2024-09-13T23:43:34.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4d544f40 1 -- start start 2024-09-13T23:43:34.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4d544f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 0x55ea2cb24b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.367+0000 7fea4d544f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c8cc000 con 0x55ea2c83f800 2024-09-13T23:43:34.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4c4fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 0x55ea2cb24b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4c4fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 0x55ea2cb24b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52936/0 (socket says 172.21.15.167:52936) 2024-09-13T23:43:34.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4c4fe700 1 -- 172.21.15.167:0/2278202004 learned_addr learned my addr 172.21.15.167:0/2278202004 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea48cf7700 1 -- 172.21.15.167:0/1925205237 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ea2c938b00 con 0x55ea2c83f000 2024-09-13T23:43:34.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea48cf7700 1 --2- 172.21.15.167:0/1925205237 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cbf0000 0x55ea2cb25080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea48cf7700 1 -- 172.21.15.167:0/1925205237 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ea2c8a5180 con 0x55ea2c83f000 2024-09-13T23:43:34.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4c4fe700 1 -- 172.21.15.167:0/2278202004 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea2c850000 con 0x55ea2c83f800 2024-09-13T23:43:34.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4bcfd700 1 --2- 172.21.15.167:0/1925205237 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cbf0000 0x55ea2cb25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4c4fe700 1 --2- 172.21.15.167:0/2278202004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 0x55ea2cb24b00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x55ea2d6b76b0 tx=0x55ea2c943f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db04497680d850e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea464f2700 1 -- 172.21.15.167:0/2278202004 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2c8e38c0 con 0x55ea2c83f800 2024-09-13T23:43:34.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea464f2700 1 -- 172.21.15.167:0/2278202004 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ea2cba6000 con 0x55ea2c83f800 2024-09-13T23:43:34.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea464f2700 1 -- 172.21.15.167:0/2278202004 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2c8e34a0 con 0x55ea2c83f800 2024-09-13T23:43:34.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4d544f40 1 -- 172.21.15.167:0/2278202004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 msgr2=0x55ea2cb24b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4bcfd700 1 --2- 172.21.15.167:0/1925205237 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cbf0000 0x55ea2cb25080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55ea2d6b6ab0 tx=0x55ea2cb94360 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4d544f40 1 --2- 172.21.15.167:0/2278202004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 0x55ea2cb24b00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x55ea2d6b76b0 tx=0x55ea2c943f20 comp rx=0 tx=0).stop 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4d544f40 1 -- 172.21.15.167:0/2278202004 shutdown_connections 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4d544f40 1 --2- 172.21.15.167:0/2278202004 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 0x55ea2cb24b00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4d544f40 1 -- 172.21.15.167:0/2278202004 >> 172.21.15.167:0/2278202004 conn(0x55ea2c83f400 msgr2=0x55ea2c93f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4d544f40 1 -- 172.21.15.167:0/2278202004 shutdown_connections 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4d544f40 1 -- 172.21.15.167:0/2278202004 wait complete. 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4d544f40 1 Processor -- start 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4d544f40 1 -- start start 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4d544f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 0x55ea2cb24b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4d544f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c864640 con 0x55ea2c83f800 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4c4fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 0x55ea2cb24b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4c4fe700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 0x55ea2cb24b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52952/0 (socket says 172.21.15.167:52952) 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 learned_addr learned my addr 172.21.15.167:0/4131136947 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.371+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea2c850ea0 con 0x55ea2c83f800 2024-09-13T23:43:34.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea4c4fe700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 0x55ea2cb24b00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x55ea2d6b6180 tx=0x55ea2cb940c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea44cef700 1 -- 172.21.15.167:0/4131136947 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2c8e3b80 con 0x55ea2c83f800 2024-09-13T23:43:34.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ea2c850820 con 0x55ea2c83f800 2024-09-13T23:43:34.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea44cef700 1 -- 172.21.15.167:0/4131136947 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ea2cba6180 con 0x55ea2c83f800 2024-09-13T23:43:34.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea44cef700 1 -- 172.21.15.167:0/4131136947 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2c8e38c0 con 0x55ea2c83f800 2024-09-13T23:43:34.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea2c8509c0 con 0x55ea2c83f800 2024-09-13T23:43:34.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea44cef700 1 -- 172.21.15.167:0/4131136947 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ea2c938dc0 con 0x55ea2c83f800 2024-09-13T23:43:34.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea44cef700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cbf0c00 0x55ea2cb25600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea44cef700 1 -- 172.21.15.167:0/4131136947 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ea2c8a5340 con 0x55ea2c83f800 2024-09-13T23:43:34.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea4b4fc700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cbf0c00 0x55ea2cb25600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea4d544f40 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ea2cbf1000 0x55ea2cb25b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1400 con 0x55ea2cbf1000 2024-09-13T23:43:34.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea4b4fc700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cbf0c00 0x55ea2cb25600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55ea2cb1f920 tx=0x55ea2c943800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.375+0000 7fea4c4fe700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ea2cbf1000 0x55ea2cb25b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4c4fe700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ea2cbf1000 0x55ea2cb25b80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 20 realm 2024-09-13T23:43:34.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:34.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1c00 con 0x55ea2cbf1000 2024-09-13T23:43:34.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ea2cbf1400 0x55ea2cb26100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1400 2024-09-13T23:43:34.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4bcfd700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ea2cbf1400 0x55ea2cb26100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4bcfd700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ea2cbf1400 0x55ea2cb26100 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:34.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.379+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1c00 con 0x55ea2cbf1400 2024-09-13T23:43:34.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4d544f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:34.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4d544f40 20 searching for the correct realm 2024-09-13T23:43:34.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4d544f40 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ea2cb5e000 0x55ea2cb26680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5e400 con 0x55ea2cb5e000 2024-09-13T23:43:34.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4b4fc700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ea2cb5e000 0x55ea2cb26680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4b4fc700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ea2cb5e000 0x55ea2cb26680 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5e400 con 0x55ea2cbf1400 2024-09-13T23:43:34.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4bcfd700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ea2cb5e400 0x55ea2cb26c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e400 2024-09-13T23:43:34.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4c4fe700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ea2cb5e400 0x55ea2cb26c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4c4fe700 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ea2cb5e400 0x55ea2cb26c00 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.383+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e000 2024-09-13T23:43:34.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1400 2024-09-13T23:43:34.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1400 2024-09-13T23:43:34.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.387+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e400 2024-09-13T23:43:34.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e400 2024-09-13T23:43:34.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1400 2024-09-13T23:43:34.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e000 2024-09-13T23:43:34.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1400 2024-09-13T23:43:34.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e000 2024-09-13T23:43:34.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.391+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e000 2024-09-13T23:43:34.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e400 2024-09-13T23:43:34.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e400 2024-09-13T23:43:34.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.395+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e000 2024-09-13T23:43:34.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.399+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.399+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e400 2024-09-13T23:43:34.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.399+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.399+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1400 2024-09-13T23:43:34.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.399+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.399+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1400 2024-09-13T23:43:34.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.399+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.399+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e000 2024-09-13T23:43:34.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.399+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.399+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1400 2024-09-13T23:43:34.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.399+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.399+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1400 2024-09-13T23:43:34.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:34.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:34.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:34.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:34.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5e800 con 0x55ea2cbf1000 2024-09-13T23:43:34.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5ec00 con 0x55ea2cbf1400 2024-09-13T23:43:34.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:34.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5f000 con 0x55ea2cbf1400 2024-09-13T23:43:34.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:34.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 zone default found 2024-09-13T23:43:34.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 4 Realm: () 2024-09-13T23:43:34.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:34.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:34.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:34.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 zonegroup default 2024-09-13T23:43:34.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5f400 con 0x55ea2cb5e400 2024-09-13T23:43:34.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5f800 con 0x55ea2cb5e000 2024-09-13T23:43:34.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.403+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.407+0000 7fea4d544f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.407+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.407+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5fc00 con 0x55ea2cb5e000 2024-09-13T23:43:34.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.407+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.407+0000 7fea4d544f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.407+0000 7fea4d544f40 20 started sync module instance, tier type = 2024-09-13T23:43:34.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.407+0000 7fea4d544f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:34.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.407+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5f800 con 0x55ea2cbf1400 2024-09-13T23:43:34.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.407+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.407+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94464260553472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5fc00 con 0x55ea2cbf1400 2024-09-13T23:43:34.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.407+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5ec00 con 0x55ea2cbf1000 2024-09-13T23:43:34.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.411+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.411+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94464260553472] v33'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.411+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94464260554624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5f000 con 0x55ea2cbf1000 2024-09-13T23:43:34.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.411+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5f800 con 0x55ea2cbf1400 2024-09-13T23:43:34.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.415+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94464260554624] v33'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.415+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.415+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94464260555776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1c00 con 0x55ea2cbf1400 2024-09-13T23:43:34.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.415+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cb5e400 2024-09-13T23:43:34.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.415+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94464260555776] v33'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.419+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.419+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94464260556928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5e800 con 0x55ea2cb5e400 2024-09-13T23:43:34.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.419+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5f400 con 0x55ea2cbf1400 2024-09-13T23:43:34.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.419+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94464260556928] v33'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.419+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.419+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94464260558080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5ec00 con 0x55ea2cbf1400 2024-09-13T23:43:34.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.419+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5f000 con 0x55ea2cb5e000 2024-09-13T23:43:34.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.423+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.423+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94464260559232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2ccb0000 con 0x55ea2cb5e000 2024-09-13T23:43:34.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.423+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2ccb0400 con 0x55ea2cb5e000 2024-09-13T23:43:34.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.423+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94464260558080] v33'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.423+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.423+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94464260560384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2ccb0800 con 0x55ea2cb5e000 2024-09-13T23:43:34.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.423+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2ccb0c00 con 0x55ea2cb5e400 2024-09-13T23:43:34.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.423+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94464260559232] v33'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94464260560384] v33'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'147 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94464260561536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2ccb1000 con 0x55ea2cb5e400 2024-09-13T23:43:34.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4d544f40 20 add_watcher() i=0 2024-09-13T23:43:34.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4d544f40 20 add_watcher() i=1 2024-09-13T23:43:34.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4d544f40 20 add_watcher() i=2 2024-09-13T23:43:34.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4d544f40 20 add_watcher() i=3 2024-09-13T23:43:34.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4d544f40 20 add_watcher() i=4 2024-09-13T23:43:34.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4d544f40 20 add_watcher() i=5 2024-09-13T23:43:34.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4d544f40 20 add_watcher() i=6 2024-09-13T23:43:34.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94464260561536] v33'148 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4d544f40 20 add_watcher() i=7 2024-09-13T23:43:34.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4d544f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:34.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.427+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2ccb1400 con 0x55ea2cbf1000 2024-09-13T23:43:34.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94464262234112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2ccb1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94464262234112] v33'144 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea32ccb700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:34.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea4d544f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:34.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea4d544f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:34.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea4d544f40 1 Processor -- start 2024-09-13T23:43:34.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea4d544f40 1 -- start start 2024-09-13T23:43:34.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea4d544f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 0x55ea2cb27180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea4d544f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c864780 con 0x55ea2ccb0c00 2024-09-13T23:43:34.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea4b4fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 0x55ea2cb27180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea4b4fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 0x55ea2cb27180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52966/0 (socket says 172.21.15.167:52966) 2024-09-13T23:43:34.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.431+0000 7fea4b4fc700 1 -- 172.21.15.167:0/3870789976 learned_addr learned my addr 172.21.15.167:0/3870789976 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4b4fc700 1 -- 172.21.15.167:0/3870789976 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea2c850b60 con 0x55ea2ccb0c00 2024-09-13T23:43:34.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4b4fc700 1 --2- 172.21.15.167:0/3870789976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 0x55ea2cb27180 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55ea2cc37800 tx=0x55ea2c943860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33156ef13bf85e02 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea324ca700 1 -- 172.21.15.167:0/3870789976 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2c8e3760 con 0x55ea2ccb0c00 2024-09-13T23:43:34.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea324ca700 1 -- 172.21.15.167:0/3870789976 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ea2cba6600 con 0x55ea2ccb0c00 2024-09-13T23:43:34.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea324ca700 1 -- 172.21.15.167:0/3870789976 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2ccca000 con 0x55ea2ccb0c00 2024-09-13T23:43:34.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4d544f40 1 -- 172.21.15.167:0/3870789976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 msgr2=0x55ea2cb27180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4d544f40 1 --2- 172.21.15.167:0/3870789976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 0x55ea2cb27180 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x55ea2cc37800 tx=0x55ea2c943860 comp rx=0 tx=0).stop 2024-09-13T23:43:34.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4d544f40 1 -- 172.21.15.167:0/3870789976 shutdown_connections 2024-09-13T23:43:34.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4d544f40 1 --2- 172.21.15.167:0/3870789976 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 0x55ea2cb27180 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4d544f40 1 -- 172.21.15.167:0/3870789976 >> 172.21.15.167:0/3870789976 conn(0x55ea2ccb1c00 msgr2=0x55ea2c940000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4d544f40 1 -- 172.21.15.167:0/3870789976 shutdown_connections 2024-09-13T23:43:34.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4d544f40 1 -- 172.21.15.167:0/3870789976 wait complete. 2024-09-13T23:43:34.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4d544f40 1 Processor -- start 2024-09-13T23:43:34.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4d544f40 1 -- start start 2024-09-13T23:43:34.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4d544f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 0x55ea2cb27180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4d544f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea2c8cc000 con 0x55ea2ccb0c00 2024-09-13T23:43:34.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4b4fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 0x55ea2cb27180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4b4fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 0x55ea2cb27180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52982/0 (socket says 172.21.15.167:52982) 2024-09-13T23:43:34.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.435+0000 7fea4b4fc700 1 -- 172.21.15.167:0/2798739354 learned_addr learned my addr 172.21.15.167:0/2798739354 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea4b4fc700 1 -- 172.21.15.167:0/2798739354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea2c850d00 con 0x55ea2ccb0c00 2024-09-13T23:43:34.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea4b4fc700 1 --2- 172.21.15.167:0/2798739354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 0x55ea2cb27180 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55ea2ccbe7b0 tx=0x55ea2ccc0300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea30cc7700 1 -- 172.21.15.167:0/2798739354 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2ccca160 con 0x55ea2ccb0c00 2024-09-13T23:43:34.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea30cc7700 1 -- 172.21.15.167:0/2798739354 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ea2cba6300 con 0x55ea2ccb0c00 2024-09-13T23:43:34.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea30cc7700 1 -- 172.21.15.167:0/2798739354 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ea2ccca2c0 con 0x55ea2ccb0c00 2024-09-13T23:43:34.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea4d544f40 1 -- 172.21.15.167:0/2798739354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ea2c850340 con 0x55ea2ccb0c00 2024-09-13T23:43:34.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea4d544f40 1 -- 172.21.15.167:0/2798739354 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea2c851d40 con 0x55ea2ccb0c00 2024-09-13T23:43:34.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea30cc7700 1 -- 172.21.15.167:0/2798739354 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ea2c939b80 con 0x55ea2ccb0c00 2024-09-13T23:43:34.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea30cc7700 1 --2- 172.21.15.167:0/2798739354 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cd3c000 0x55ea2cd42000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea30cc7700 1 -- 172.21.15.167:0/2798739354 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ea2c8a5500 con 0x55ea2ccb0c00 2024-09-13T23:43:34.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea4c4fe700 1 --2- 172.21.15.167:0/2798739354 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cd3c000 0x55ea2cd42000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea4d544f40 5 note: GC not initialized 2024-09-13T23:43:34.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.439+0000 7fea4c4fe700 1 --2- 172.21.15.167:0/2798739354 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cd3c000 0x55ea2cd42000 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55ea2cc37da0 tx=0x55ea2cb957a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea2e4c2700 20 reqs_thread_entry: start 2024-09-13T23:43:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea2c4be700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:04 2024 2024-09-13T23:43:34.452 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2ccb0000 con 0x55ea2cb5e400 2024-09-13T23:43:34.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea2bc9c700 20 reqs_thread_entry: start 2024-09-13T23:43:34.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:34.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2ccb0400 con 0x55ea2cb5e000 2024-09-13T23:43:34.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:34.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-13T23:43:34.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5e800 con 0x55ea2cb5e400 2024-09-13T23:43:34.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:34.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.443+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5f000 con 0x55ea2cb5e000 2024-09-13T23:43:34.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(65 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:34.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 20 couldn't put binfo cache entry, might have raced with data changes 2024-09-13T23:43:34.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 4.a 4:517e2946:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1400 2024-09-13T23:43:34.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 4.8 4:10b47b19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cd3c400 con 0x55ea2cb5e000 2024-09-13T23:43:34.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 4.2e 4:75bb53d2:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cd3c800 con 0x55ea2cb5e000 2024-09-13T23:43:34.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 4.26 4:6424e2c9:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cd3cc00 con 0x55ea2cbf1000 2024-09-13T23:43:34.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 4.c 4:32a29db1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cd3d000 con 0x55ea2cbf1000 2024-09-13T23:43:34.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 4.28 4:162fa60a:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cd3d400 con 0x55ea2cbf1400 2024-09-13T23:43:34.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 4.2c 4:355fcb42:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cd3d800 con 0x55ea2cbf1000 2024-09-13T23:43:34.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 4.2f 4:f64e09ba:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cd3dc00 con 0x55ea2cbf1000 2024-09-13T23:43:34.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:74 4.3a 4:5ee337b5:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1c00 con 0x55ea2cb5e000 2024-09-13T23:43:34.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:75 4.2c 4:3559f32d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cbf1800 con 0x55ea2cbf1000 2024-09-13T23:43:34.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 4.1 4:8131d739:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5f000 con 0x55ea2cb5e400 2024-09-13T23:43:34.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(67 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(68 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(66 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(71 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.5 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(74 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.8 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(69 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.3 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(73 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.7 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(70 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(72 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.6 [call out=124b] v0'0 uv8 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(76 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(75 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1.9 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea32ccb700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:34.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.447+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94464262234112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5fc00 con 0x55ea2cbf1000 2024-09-13T23:43:34.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.451+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94464262234112] v33'145 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.451+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94464260553472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5f800 con 0x55ea2cbf1400 2024-09-13T23:43:34.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.455+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94464260553472] v33'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.455+0000 7fea4d544f40 20 remove_watcher() i=0 2024-09-13T23:43:34.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.455+0000 7fea4d544f40 2 removed watcher, disabling cache 2024-09-13T23:43:34.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.455+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94464260554624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5f400 con 0x55ea2cbf1000 2024-09-13T23:43:34.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.455+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94464260554624] v33'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cbf1000 2024-09-13T23:43:34.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.455+0000 7fea4d544f40 20 remove_watcher() i=1 2024-09-13T23:43:34.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.455+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94464260555776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cb5ec00 con 0x55ea2cbf1400 2024-09-13T23:43:34.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.459+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94464260555776] v33'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.459+0000 7fea4d544f40 20 remove_watcher() i=2 2024-09-13T23:43:34.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.459+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94464260556928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cd3c400 con 0x55ea2cb5e400 2024-09-13T23:43:34.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.459+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94464260556928] v33'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.459+0000 7fea4d544f40 20 remove_watcher() i=3 2024-09-13T23:43:34.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.459+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94464260558080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cd3c800 con 0x55ea2cbf1400 2024-09-13T23:43:34.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.463+0000 7fea4bcfd700 1 -- 172.21.15.167:0/4131136947 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94464260558080] v33'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b200 con 0x55ea2cbf1400 2024-09-13T23:43:34.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.463+0000 7fea4d544f40 20 remove_watcher() i=4 2024-09-13T23:43:34.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.463+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94464260559232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2ccb0400 con 0x55ea2cb5e000 2024-09-13T23:43:34.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.463+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94464260559232] v33'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.463+0000 7fea4d544f40 20 remove_watcher() i=5 2024-09-13T23:43:34.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.463+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94464260560384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cd3cc00 con 0x55ea2cb5e000 2024-09-13T23:43:34.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.467+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94464260560384] v33'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9b440 con 0x55ea2cb5e000 2024-09-13T23:43:34.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.467+0000 7fea4d544f40 20 remove_watcher() i=6 2024-09-13T23:43:34.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.467+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94464260561536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ea2cd3d000 con 0x55ea2cb5e400 2024-09-13T23:43:34.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4c4fe700 1 -- 172.21.15.167:0/4131136947 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94464260561536] v33'149 uv147 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ea2cc9ab40 con 0x55ea2cb5e400 2024-09-13T23:43:34.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 20 remove_watcher() i=7 2024-09-13T23:43:34.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ea2cbf1400 msgr2=0x55ea2cb26100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ea2cbf1400 0x55ea2cb26100 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ea2cbf1000 msgr2=0x55ea2cb25b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ea2cbf1000 0x55ea2cb25b80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ea2cb5e400 msgr2=0x55ea2cb26c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ea2cb5e400 0x55ea2cb26c00 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ea2cb5e000 msgr2=0x55ea2cb26680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ea2cb5e000 0x55ea2cb26680 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cbf0c00 msgr2=0x55ea2cb25600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cbf0c00 0x55ea2cb25600 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55ea2cb1f920 tx=0x55ea2c943800 comp rx=0 tx=0).stop 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 msgr2=0x55ea2cb24b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 0x55ea2cb24b00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x55ea2d6b6180 tx=0x55ea2cb940c0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4b4fc700 1 -- 172.21.15.167:0/4131136947 reap_dead start 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 shutdown_connections 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/4131136947 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f800 0x55ea2cb24b00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 >> 172.21.15.167:0/4131136947 conn(0x55ea2c83f400 msgr2=0x55ea2c93f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 shutdown_connections 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/4131136947 wait complete. 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/2798739354 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cd3c000 msgr2=0x55ea2cd42000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/2798739354 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cd3c000 0x55ea2cd42000 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x55ea2cc37da0 tx=0x55ea2cb957a0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/2798739354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 msgr2=0x55ea2cb27180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/2798739354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 0x55ea2cb27180 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x55ea2ccbe7b0 tx=0x55ea2ccc0300 comp rx=0 tx=0).stop 2024-09-13T23:43:34.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/2798739354 shutdown_connections 2024-09-13T23:43:34.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/2798739354 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cd3c000 0x55ea2cd42000 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/2798739354 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2ccb0c00 0x55ea2cb27180 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/2798739354 >> 172.21.15.167:0/2798739354 conn(0x55ea2ccb1c00 msgr2=0x55ea2c940000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/2798739354 shutdown_connections 2024-09-13T23:43:34.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/2798739354 wait complete. 2024-09-13T23:43:34.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/1925205237 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cbf0000 msgr2=0x55ea2cb25080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/1925205237 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cbf0000 0x55ea2cb25080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x55ea2d6b6ab0 tx=0x55ea2cb94360 comp rx=0 tx=0).stop 2024-09-13T23:43:34.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/1925205237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 msgr2=0x55ea2cb24580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/1925205237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 0x55ea2cb24580 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x55ea2c99bfb0 tx=0x55ea2c943a40 comp rx=0 tx=0).stop 2024-09-13T23:43:34.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/1925205237 shutdown_connections 2024-09-13T23:43:34.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/1925205237 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ea2cbf0000 0x55ea2cb25080 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 --2- 172.21.15.167:0/1925205237 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ea2c83f000 0x55ea2cb24580 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/1925205237 >> 172.21.15.167:0/1925205237 conn(0x55ea2c83ec00 msgr2=0x55ea2c93f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/1925205237 shutdown_connections 2024-09-13T23:43:34.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.471+0000 7fea4d544f40 1 -- 172.21.15.167:0/1925205237 wait complete. 2024-09-13T23:43:34.486 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:34.486 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo", 2024-09-13T23:43:34.486 INFO:teuthology.orchestra.run.smithi167.stdout: "num_shards": 11, 2024-09-13T23:43:34.487 INFO:teuthology.orchestra.run.smithi167.stdout: "tenant": "", 2024-09-13T23:43:34.487 INFO:teuthology.orchestra.run.smithi167.stdout: "zonegroup": "363ea8c8-5049-424d-855a-cd6979bf7acf", 2024-09-13T23:43:34.487 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_rule": "default-placement", 2024-09-13T23:43:34.487 INFO:teuthology.orchestra.run.smithi167.stdout: "explicit_placement": { 2024-09-13T23:43:34.487 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": "", 2024-09-13T23:43:34.487 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "", 2024-09-13T23:43:34.487 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": "" 2024-09-13T23:43:34.488 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:34.488 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1", 2024-09-13T23:43:34.488 INFO:teuthology.orchestra.run.smithi167.stdout: "marker": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1", 2024-09-13T23:43:34.488 INFO:teuthology.orchestra.run.smithi167.stdout: "index_type": "Normal", 2024-09-13T23:43:34.488 INFO:teuthology.orchestra.run.smithi167.stdout: "versioned": false, 2024-09-13T23:43:34.488 INFO:teuthology.orchestra.run.smithi167.stdout: "versioning_enabled": false, 2024-09-13T23:43:34.489 INFO:teuthology.orchestra.run.smithi167.stdout: "object_lock_enabled": false, 2024-09-13T23:43:34.489 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_enabled": false, 2024-09-13T23:43:34.489 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:43:34.489 INFO:teuthology.orchestra.run.smithi167.stdout: "ver": "0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1", 2024-09-13T23:43:34.489 INFO:teuthology.orchestra.run.smithi167.stdout: "master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0", 2024-09-13T23:43:34.490 INFO:teuthology.orchestra.run.smithi167.stdout: "mtime": "2024-09-13T23:43:33.681243Z", 2024-09-13T23:43:34.490 INFO:teuthology.orchestra.run.smithi167.stdout: "creation_time": "2024-09-13T23:43:29.849424Z", 2024-09-13T23:43:34.490 INFO:teuthology.orchestra.run.smithi167.stdout: "max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#", 2024-09-13T23:43:34.490 INFO:teuthology.orchestra.run.smithi167.stdout: "usage": { 2024-09-13T23:43:34.490 INFO:teuthology.orchestra.run.smithi167.stdout: "rgw.main": { 2024-09-13T23:43:34.491 INFO:teuthology.orchestra.run.smithi167.stdout: "size": 0, 2024-09-13T23:43:34.491 INFO:teuthology.orchestra.run.smithi167.stdout: "size_actual": 0, 2024-09-13T23:43:34.491 INFO:teuthology.orchestra.run.smithi167.stdout: "size_utilized": 0, 2024-09-13T23:43:34.491 INFO:teuthology.orchestra.run.smithi167.stdout: "size_kb": 0, 2024-09-13T23:43:34.491 INFO:teuthology.orchestra.run.smithi167.stdout: "size_kb_actual": 0, 2024-09-13T23:43:34.491 INFO:teuthology.orchestra.run.smithi167.stdout: "size_kb_utilized": 0, 2024-09-13T23:43:34.491 INFO:teuthology.orchestra.run.smithi167.stdout: "num_objects": 0 2024-09-13T23:43:34.492 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:34.492 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:34.492 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:34.492 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:34.492 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:34.492 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:34.493 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:34.493 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:34.493 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:34.493 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:34.495 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '363ea8c8-5049-424d-855a-cd6979bf7acf', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1', 'marker': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-13T23:43:33.681243Z', 'creation_time': '2024-09-13T23:43:29.849424Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-09-13T23:43:34.495 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'list'] 2024-09-13T23:43:34.495 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'list'] 2024-09-13T23:43:34.495 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log list 2024-09-13T23:43:34.580 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:34.580 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:34.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.571+0000 7f9e85f0df40 1 Processor -- start 2024-09-13T23:43:34.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.571+0000 7f9e85f0df40 1 -- start start 2024-09-13T23:43:34.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.571+0000 7f9e85f0df40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x557792ae3000 0x557792dc8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.571+0000 7f9e85f0df40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x557792ba7500 con 0x557792ae3400 2024-09-13T23:43:34.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.571+0000 7f9e85f0df40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x557792b70a00 con 0x557792ae3000 2024-09-13T23:43:34.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.571+0000 7f9e846c6700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x557792ae3000 0x557792dc8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.571+0000 7f9e83ec5700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x557792ae3400 0x557792be2800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45566/0 (socket says 172.21.15.167:45566) 2024-09-13T23:43:34.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.571+0000 7f9e846c6700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x557792ae3000 0x557792dc8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52996/0 (socket says 172.21.15.167:52996) 2024-09-13T23:43:34.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.571+0000 7f9e83ec5700 1 -- 172.21.15.167:0/2081313040 learned_addr learned my addr 172.21.15.167:0/2081313040 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:34.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.571+0000 7f9e846c6700 1 -- 172.21.15.167:0/2081313040 >> v1:172.21.15.12:6789/0 conn(0x557792ae3400 legacy=0x557792be2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.571+0000 7f9e846c6700 1 -- 172.21.15.167:0/2081313040 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557792af5d40 con 0x557792ae3000 2024-09-13T23:43:34.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.571+0000 7f9e836c4700 1 -- 172.21.15.167:0/2081313040 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4096455409 0 0) 0x557792c3e000 con 0x557792ae3400 2024-09-13T23:43:34.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e846c6700 1 --2- 172.21.15.167:0/2081313040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3000 0x557792dc8580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x557792dc3a70 tx=0x557792be7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ff95ae277ba036 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e836c4700 1 -- 172.21.15.167:0/2081313040 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792b874a0 con 0x557792ae3000 2024-09-13T23:43:34.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e836c4700 1 -- 172.21.15.167:0/2081313040 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557792ba7800 con 0x557792ae3000 2024-09-13T23:43:34.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e836c4700 1 -- 172.21.15.167:0/2081313040 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792b87600 con 0x557792ae3000 2024-09-13T23:43:34.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2081313040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3000 msgr2=0x557792dc8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/2081313040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3000 0x557792dc8580 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x557792dc3a70 tx=0x557792be7920 comp rx=0 tx=0).stop 2024-09-13T23:43:34.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2081313040 shutdown_connections 2024-09-13T23:43:34.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/2081313040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3000 0x557792dc8580 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2081313040 >> 172.21.15.167:0/2081313040 conn(0x557792ae2c00 msgr2=0x557792be2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2081313040 shutdown_connections 2024-09-13T23:43:34.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2081313040 wait complete. 2024-09-13T23:43:34.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e85f0df40 1 Processor -- start 2024-09-13T23:43:34.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e85f0df40 1 -- start start 2024-09-13T23:43:34.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e85f0df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3000 0x557792dc8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e85f0df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557792b70500 con 0x557792ae3000 2024-09-13T23:43:34.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e846c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3000 0x557792dc8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e846c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3000 0x557792dc8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53008/0 (socket says 172.21.15.167:53008) 2024-09-13T23:43:34.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e846c6700 1 -- 172.21.15.167:0/1860291073 learned_addr learned my addr 172.21.15.167:0/1860291073 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e846c6700 1 -- 172.21.15.167:0/1860291073 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557792af5ba0 con 0x557792ae3000 2024-09-13T23:43:34.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e846c6700 1 --2- 172.21.15.167:0/1860291073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3000 0x557792dc8580 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x5577938ade00 tx=0x557792be7800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe064db8a7c2b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.575+0000 7f9e82ec3700 1 -- 172.21.15.167:0/1860291073 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792b87760 con 0x557792ae3000 2024-09-13T23:43:34.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e82ec3700 1 -- 172.21.15.167:0/1860291073 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557792ba7b00 con 0x557792ae3000 2024-09-13T23:43:34.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e82ec3700 1 -- 172.21.15.167:0/1860291073 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792b874a0 con 0x557792ae3000 2024-09-13T23:43:34.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 -- 172.21.15.167:0/1860291073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3000 msgr2=0x557792dc8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/1860291073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3000 0x557792dc8580 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x5577938ade00 tx=0x557792be7800 comp rx=0 tx=0).stop 2024-09-13T23:43:34.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 -- 172.21.15.167:0/1860291073 shutdown_connections 2024-09-13T23:43:34.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/1860291073 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3000 0x557792dc8580 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 -- 172.21.15.167:0/1860291073 >> 172.21.15.167:0/1860291073 conn(0x557792ae2c00 msgr2=0x557792be3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 -- 172.21.15.167:0/1860291073 shutdown_connections 2024-09-13T23:43:34.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 -- 172.21.15.167:0/1860291073 wait complete. 2024-09-13T23:43:34.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 Processor -- start 2024-09-13T23:43:34.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 -- start start 2024-09-13T23:43:34.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3400 0x557792dc8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557792b703c0 con 0x557792ae3400 2024-09-13T23:43:34.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e846c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3400 0x557792dc8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e846c6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3400 0x557792dc8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53020/0 (socket says 172.21.15.167:53020) 2024-09-13T23:43:34.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e846c6700 1 -- 172.21.15.167:0/2843038813 learned_addr learned my addr 172.21.15.167:0/2843038813 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e846c6700 1 -- 172.21.15.167:0/2843038813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557792af5a00 con 0x557792ae3400 2024-09-13T23:43:34.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e846c6700 1 --2- 172.21.15.167:0/2843038813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3400 0x557792dc8b00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x5577938ade90 tx=0x557792be7920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e816c0700 1 -- 172.21.15.167:0/2843038813 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792b87600 con 0x557792ae3400 2024-09-13T23:43:34.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e816c0700 1 -- 172.21.15.167:0/2843038813 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557792ba7c80 con 0x557792ae3400 2024-09-13T23:43:34.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2843038813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557792af5860 con 0x557792ae3400 2024-09-13T23:43:34.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e816c0700 1 -- 172.21.15.167:0/2843038813 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792b878c0 con 0x557792ae3400 2024-09-13T23:43:34.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.579+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2843038813 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557792af56c0 con 0x557792ae3400 2024-09-13T23:43:34.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e85f0df40 1 Processor -- start 2024-09-13T23:43:34.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e85f0df40 1 -- start start 2024-09-13T23:43:34.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e85f0df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 0x557792dc9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e85f0df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557792b70000 con 0x557792ae3c00 2024-09-13T23:43:34.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e816c0700 1 -- 172.21.15.167:0/2843038813 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x557792bdcb00 con 0x557792ae3400 2024-09-13T23:43:34.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e84ec7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 0x557792dc9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e816c0700 1 --2- 172.21.15.167:0/2843038813 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792e84000 0x557792dca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e84ec7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 0x557792dc9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53026/0 (socket says 172.21.15.167:53026) 2024-09-13T23:43:34.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e84ec7700 1 -- 172.21.15.167:0/3281810533 learned_addr learned my addr 172.21.15.167:0/3281810533 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e816c0700 1 -- 172.21.15.167:0/2843038813 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x557792b49180 con 0x557792ae3400 2024-09-13T23:43:34.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e846c6700 1 --2- 172.21.15.167:0/2843038813 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792e84000 0x557792dca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e84ec7700 1 -- 172.21.15.167:0/3281810533 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557792af4000 con 0x557792ae3c00 2024-09-13T23:43:34.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e846c6700 1 --2- 172.21.15.167:0/2843038813 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792e84000 0x557792dca100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x557792e0b620 tx=0x557792e46c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e84ec7700 1 --2- 172.21.15.167:0/3281810533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 0x557792dc9080 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x557792dc3f80 tx=0x557792e46b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe56abaadee9f774 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e7eebb700 1 -- 172.21.15.167:0/3281810533 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792b87600 con 0x557792ae3c00 2024-09-13T23:43:34.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e7eebb700 1 -- 172.21.15.167:0/3281810533 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557792e92000 con 0x557792ae3c00 2024-09-13T23:43:34.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e7eebb700 1 -- 172.21.15.167:0/3281810533 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792b878c0 con 0x557792ae3c00 2024-09-13T23:43:34.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3281810533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 msgr2=0x557792dc9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/3281810533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 0x557792dc9080 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x557792dc3f80 tx=0x557792e46b40 comp rx=0 tx=0).stop 2024-09-13T23:43:34.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3281810533 shutdown_connections 2024-09-13T23:43:34.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.583+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/3281810533 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 0x557792dc9080 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3281810533 >> 172.21.15.167:0/3281810533 conn(0x557792ae3800 msgr2=0x557792be3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3281810533 shutdown_connections 2024-09-13T23:43:34.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3281810533 wait complete. 2024-09-13T23:43:34.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e85f0df40 1 Processor -- start 2024-09-13T23:43:34.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e85f0df40 1 -- start start 2024-09-13T23:43:34.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e85f0df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 0x557792dc9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e85f0df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557792b08640 con 0x557792ae3c00 2024-09-13T23:43:34.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e84ec7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 0x557792dc9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e84ec7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 0x557792dc9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53040/0 (socket says 172.21.15.167:53040) 2024-09-13T23:43:34.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 learned_addr learned my addr 172.21.15.167:0/173588780 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557792af4ea0 con 0x557792ae3c00 2024-09-13T23:43:34.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e84ec7700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 0x557792dc9080 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x557792e0a870 tx=0x557792e465a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e7d6b8700 1 -- 172.21.15.167:0/173588780 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792b87760 con 0x557792ae3c00 2024-09-13T23:43:34.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e7d6b8700 1 -- 172.21.15.167:0/173588780 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557792e92180 con 0x557792ae3c00 2024-09-13T23:43:34.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557792af4820 con 0x557792ae3c00 2024-09-13T23:43:34.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e7d6b8700 1 -- 172.21.15.167:0/173588780 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792b87600 con 0x557792ae3c00 2024-09-13T23:43:34.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557792af49c0 con 0x557792ae3c00 2024-09-13T23:43:34.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.587+0000 7f9e85f0df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e7d6b8700 1 -- 172.21.15.167:0/173588780 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x557792bdcdc0 con 0x557792ae3c00 2024-09-13T23:43:34.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e7d6b8700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792e84c00 0x557792dca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e7d6b8700 1 -- 172.21.15.167:0/173588780 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x557792b49340 con 0x557792ae3c00 2024-09-13T23:43:34.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e83ec5700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792e84c00 0x557792dca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557792e85000 0x557792dc9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792e85400 con 0x557792e85000 2024-09-13T23:43:34.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e84ec7700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557792e85000 0x557792dc9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e83ec5700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792e84c00 0x557792dca680 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5577938ac030 tx=0x557792f3c0c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e84ec7700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557792e85000 0x557792dc9600 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 20 realm 2024-09-13T23:43:34.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792e85800 con 0x557792e85000 2024-09-13T23:43:34.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:34.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792e85c00 con 0x557792e85000 2024-09-13T23:43:34.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557792e85400 0x557792dc9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792e85800 con 0x557792e85400 2024-09-13T23:43:34.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.591+0000 7f9e846c6700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557792e85400 0x557792dc9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e846c6700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557792e85400 0x557792dc9b80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e85f0df40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:34.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e85f0df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792e85c00 con 0x557792e85400 2024-09-13T23:43:34.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e85f0df40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:34.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e85f0df40 20 searching for the correct realm 2024-09-13T23:43:34.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557792e85800 0x557792dcac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792e85c00 con 0x557792e85800 2024-09-13T23:43:34.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e83ec5700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557792e85800 0x557792dcac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e83ec5700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557792e85800 0x557792dcac00 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.595+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792e85c00 con 0x557792e85400 2024-09-13T23:43:34.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e846c6700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557792e85c00 0x557792dc8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85c00 2024-09-13T23:43:34.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e84ec7700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557792e85c00 0x557792dc8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e84ec7700 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557792e85c00 0x557792dc8580 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85000 2024-09-13T23:43:34.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85000 2024-09-13T23:43:34.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85800 2024-09-13T23:43:34.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.599+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85000 2024-09-13T23:43:34.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85400 2024-09-13T23:43:34.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85400 2024-09-13T23:43:34.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85000 2024-09-13T23:43:34.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85c00 2024-09-13T23:43:34.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85000 2024-09-13T23:43:34.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85c00 2024-09-13T23:43:34.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.603+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85400 2024-09-13T23:43:34.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.607+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.607+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85800 2024-09-13T23:43:34.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.607+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.607+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85000 2024-09-13T23:43:34.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.607+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.607+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85400 2024-09-13T23:43:34.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.607+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.607+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85800 2024-09-13T23:43:34.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.607+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.607+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85000 2024-09-13T23:43:34.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.607+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.607+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85000 2024-09-13T23:43:34.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.611+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.611+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85000 2024-09-13T23:43:34.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.611+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.611+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85800 2024-09-13T23:43:34.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.611+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.611+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85c00 2024-09-13T23:43:34.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.611+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.611+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85000 2024-09-13T23:43:34.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.611+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.611+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85c00 2024-09-13T23:43:34.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.611+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.611+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85800 2024-09-13T23:43:34.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.615+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.615+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85c00 2024-09-13T23:43:34.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.615+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.615+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85400 2024-09-13T23:43:34.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.615+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.615+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85400 2024-09-13T23:43:34.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.615+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.615+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85800 2024-09-13T23:43:34.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.615+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.615+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85400 2024-09-13T23:43:34.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.615+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.615+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85400 2024-09-13T23:43:34.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:34.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:34.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:34.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:34.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c800 con 0x557792e85000 2024-09-13T23:43:34.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792e85400 2024-09-13T23:43:34.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:34.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4d000 con 0x557792e85400 2024-09-13T23:43:34.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:34.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 zone default found 2024-09-13T23:43:34.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 4 Realm: () 2024-09-13T23:43:34.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:34.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:34.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:34.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 zonegroup default 2024-09-13T23:43:34.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4d400 con 0x557792e85c00 2024-09-13T23:43:34.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.619+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4d800 con 0x557792e85800 2024-09-13T23:43:34.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.623+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.623+0000 7f9e85f0df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.623+0000 7f9e85f0df40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.623+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4dc00 con 0x557792e85800 2024-09-13T23:43:34.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.623+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.623+0000 7f9e85f0df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.623+0000 7f9e85f0df40 20 started sync module instance, tier type = 2024-09-13T23:43:34.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.623+0000 7f9e85f0df40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:34.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.623+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4d800 con 0x557792e85400 2024-09-13T23:43:34.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.627+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.627+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93972053359360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4dc00 con 0x557792e85400 2024-09-13T23:43:34.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.627+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c400 con 0x557792e85000 2024-09-13T23:43:34.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.627+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.627+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93972053360512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85000 2024-09-13T23:43:34.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.627+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792e85400 2024-09-13T23:43:34.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.627+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 93972053359360] v33'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.631+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93972053360512] v33'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.631+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.631+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93972053361664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4d000 con 0x557792e85400 2024-09-13T23:43:34.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.631+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c800 con 0x557792e85c00 2024-09-13T23:43:34.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.635+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.635+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93972053362816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4d400 con 0x557792e85c00 2024-09-13T23:43:34.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.635+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c400 con 0x557792e85400 2024-09-13T23:43:34.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.635+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 93972053361664] v33'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.635+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 93972053362816] v33'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.635+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.635+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93972053363968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792e85400 2024-09-13T23:43:34.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.635+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4d800 con 0x557792e85800 2024-09-13T23:43:34.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.639+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.639+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93972053365120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4dc00 con 0x557792e85800 2024-09-13T23:43:34.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.639+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792e85800 2024-09-13T23:43:34.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.639+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93972053363968] v33'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.639+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93972053365120] v33'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.639+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.639+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93972053366272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4d000 con 0x557792e85800 2024-09-13T23:43:34.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.639+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c400 con 0x557792e85c00 2024-09-13T23:43:34.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.643+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93972053366272] v33'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.643+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'150 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.643+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93972053367424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4d800 con 0x557792e85c00 2024-09-13T23:43:34.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.643+0000 7f9e85f0df40 20 add_watcher() i=0 2024-09-13T23:43:34.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.643+0000 7f9e85f0df40 20 add_watcher() i=1 2024-09-13T23:43:34.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.643+0000 7f9e85f0df40 20 add_watcher() i=2 2024-09-13T23:43:34.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.643+0000 7f9e85f0df40 20 add_watcher() i=3 2024-09-13T23:43:34.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.643+0000 7f9e85f0df40 20 add_watcher() i=4 2024-09-13T23:43:34.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.643+0000 7f9e85f0df40 20 add_watcher() i=5 2024-09-13T23:43:34.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.643+0000 7f9e85f0df40 20 add_watcher() i=6 2024-09-13T23:43:34.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.647+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 93972053367424] v33'151 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.647+0000 7f9e85f0df40 20 add_watcher() i=7 2024-09-13T23:43:34.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.647+0000 7f9e85f0df40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:34.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.647+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4dc00 con 0x557792e85000 2024-09-13T23:43:34.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.647+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.647+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93972055040000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792e85000 2024-09-13T23:43:34.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.647+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93972055040000] v33'146 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.647+0000 7f9e6b694700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:34.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.647+0000 7f9e85f0df40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:34.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.647+0000 7f9e85f0df40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:34.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.647+0000 7f9e85f0df40 1 Processor -- start 2024-09-13T23:43:34.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 -- start start 2024-09-13T23:43:34.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 0x557792dcb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557792b08780 con 0x557792f4d400 2024-09-13T23:43:34.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e83ec5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 0x557792dcb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e83ec5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 0x557792dcb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53050/0 (socket says 172.21.15.167:53050) 2024-09-13T23:43:34.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3189634955 learned_addr learned my addr 172.21.15.167:0/3189634955 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3189634955 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557792af4b60 con 0x557792f4d400 2024-09-13T23:43:34.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e83ec5700 1 --2- 172.21.15.167:0/3189634955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 0x557792dcb180 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x557792e63f80 tx=0x557792f3c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416e78157162bec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e6ae93700 1 -- 172.21.15.167:0/3189634955 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792f6c000 con 0x557792f4d400 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e6ae93700 1 -- 172.21.15.167:0/3189634955 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557792e92600 con 0x557792f4d400 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e6ae93700 1 -- 172.21.15.167:0/3189634955 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792f6c160 con 0x557792f4d400 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3189634955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 msgr2=0x557792dcb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/3189634955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 0x557792dcb180 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x557792e63f80 tx=0x557792f3c060 comp rx=0 tx=0).stop 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3189634955 shutdown_connections 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/3189634955 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 0x557792dcb180 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3189634955 >> 172.21.15.167:0/3189634955 conn(0x557792f4c800 msgr2=0x557792be4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3189634955 shutdown_connections 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3189634955 wait complete. 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 Processor -- start 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 -- start start 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 0x557792dcb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.651+0000 7f9e85f0df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x557792b70000 con 0x557792f4d400 2024-09-13T23:43:34.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e83ec5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 0x557792dcb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e83ec5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 0x557792dcb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53066/0 (socket says 172.21.15.167:53066) 2024-09-13T23:43:34.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 learned_addr learned my addr 172.21.15.167:0/3940404918 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557792af4d00 con 0x557792f4d400 2024-09-13T23:43:34.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e83ec5700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 0x557792dcb180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x557792f65260 tx=0x557792f3c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e69690700 1 -- 172.21.15.167:0/3940404918 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792f6c2c0 con 0x557792f4d400 2024-09-13T23:43:34.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e69690700 1 -- 172.21.15.167:0/3940404918 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557792e92300 con 0x557792f4d400 2024-09-13T23:43:34.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557792af4340 con 0x557792f4d400 2024-09-13T23:43:34.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e69690700 1 -- 172.21.15.167:0/3940404918 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x557792f6c000 con 0x557792f4d400 2024-09-13T23:43:34.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557792af4b60 con 0x557792f4d400 2024-09-13T23:43:34.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e69690700 1 -- 172.21.15.167:0/3940404918 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x557792bddb80 con 0x557792f4d400 2024-09-13T23:43:34.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e69690700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792fe6000 0x557792fec000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e69690700 1 -- 172.21.15.167:0/3940404918 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x557792b49500 con 0x557792f4d400 2024-09-13T23:43:34.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e85f0df40 5 note: GC not initialized 2024-09-13T23:43:34.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e84ec7700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792fe6000 0x557792fec000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.655+0000 7f9e84ec7700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792fe6000 0x557792fec000 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x557792e62330 tx=0x557792e47500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e85f0df40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e66e8b700 20 reqs_thread_entry: start 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e85f0df40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e64686700 20 reqs_thread_entry: start 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557792f4dc00 0x557792dcb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 6.0 6:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792fe6400 con 0x557792f4dc00 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e64e87700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:04 2024 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e846c6700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557792f4dc00 0x557792dcb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e846c6700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557792f4dc00 0x557792dcb700 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 [pgnls start_epoch 33 out=254b] v33'34 uv18 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x557792f13200 con 0x557792f4dc00 2024-09-13T23:43:34.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 6.10 6:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792fe6400 con 0x557792f4dc00 2024-09-13T23:43:34.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 [pgnls start_epoch 33 out=131b] v33'4 uv4 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x557792f13200 con 0x557792f4dc00 2024-09-13T23:43:34.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e846c6700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557792fe6400 0x557792fec580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:3 6.8 6:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4c000 con 0x557792fe6400 2024-09-13T23:43:34.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.659+0000 7f9e83ec5700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557792fe6400 0x557792fec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e83ec5700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557792fe6400 0x557792fec580 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(3 [pgnls start_epoch 33 out=295b] v33'23 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x557792f13440 con 0x557792fe6400 2024-09-13T23:43:34.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e83ec5700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557792f4c000 0x557792fecb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:4 6.18 6:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557793052000 con 0x557792f4c000 2024-09-13T23:43:34.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e84ec7700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557792f4c000 0x557792fecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e84ec7700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557792f4c000 0x557792fecb00 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e84ec7700 1 -- 172.21.15.167:0/3940404918 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(4 [pgnls start_epoch 33 out=172b] v33'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x557792f12b40 con 0x557792f4c000 2024-09-13T23:43:34.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e84ec7700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:5 6.4 6:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557793052000 con 0x557792f4dc00 2024-09-13T23:43:34.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(5 [pgnls start_epoch 33 out=335b] v33'146 uv23 ondisk = 1) v8 ==== 144+0+335 (crc 0 0 0) 0x557792f13200 con 0x557792f4dc00 2024-09-13T23:43:34.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:6 6.14 6:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557793052000 con 0x557792f4c000 2024-09-13T23:43:34.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e84ec7700 1 -- 172.21.15.167:0/3940404918 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(6 [pgnls start_epoch 33 out=213b] v33'10 uv10 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x557792f12b40 con 0x557792f4c000 2024-09-13T23:43:34.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e84ec7700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557793052000 0x557792fed080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.663+0000 7f9e84ec7700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:7 6.c 6:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557793052000 2024-09-13T23:43:34.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.667+0000 7f9e846c6700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557793052000 0x557792fed080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.667+0000 7f9e846c6700 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557793052000 0x557792fed080 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.667+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(7 [pgnls start_epoch 33 out=254b] v33'10 uv10 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x557792f13200 con 0x557793052000 2024-09-13T23:43:34.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.667+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:8 6.1c 6:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792fe6400 2024-09-13T23:43:34.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.667+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(8 [pgnls start_epoch 33 out=295b] v33'24 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x557792f13440 con 0x557792fe6400 2024-09-13T23:43:34.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.667+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:9 6.2 6:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557793052000 2024-09-13T23:43:34.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.667+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(9 [pgnls start_epoch 33 out=213b] v33'10 uv10 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x557792f13200 con 0x557793052000 2024-09-13T23:43:34.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.667+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.12 6:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792fe6400 2024-09-13T23:43:34.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.667+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(10 [pgnls start_epoch 33 out=172b] v33'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x557792f13440 con 0x557792fe6400 2024-09-13T23:43:34.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.667+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.a 6:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792fe6400 2024-09-13T23:43:34.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.671+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(11 [pgnls start_epoch 33 out=336b] v33'27 uv19 ondisk = 1) v8 ==== 144+0+336 (crc 0 0 0) 0x557792f13440 con 0x557792fe6400 2024-09-13T23:43:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.671+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:12 6.1a 6:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792fe6400 2024-09-13T23:43:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.671+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(12 [pgnls start_epoch 33 out=131b] v33'17 uv9 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x557792f13440 con 0x557792fe6400 2024-09-13T23:43:34.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.671+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:13 6.6 6:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557793052000 2024-09-13T23:43:34.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.671+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(13 [pgnls start_epoch 33 out=254b] v33'12 uv12 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x557792f13200 con 0x557793052000 2024-09-13T23:43:34.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.671+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:14 6.16 6:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557793052000 2024-09-13T23:43:34.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.671+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(14 [pgnls start_epoch 33 out=90b] v33'17 uv9 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x557792f13200 con 0x557793052000 2024-09-13T23:43:34.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.671+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:15 6.e 6:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792fe6400 2024-09-13T23:43:34.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.675+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(15 [pgnls start_epoch 33 out=213b] v33'19 uv11 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x557792f13440 con 0x557792fe6400 2024-09-13T23:43:34.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.675+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:16 6.1e 6:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557793052000 2024-09-13T23:43:34.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.675+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(16 [pgnls start_epoch 33 out=254b] v33'10 uv10 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x557792f13200 con 0x557793052000 2024-09-13T23:43:34.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.675+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 6.1 6:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792f4dc00 2024-09-13T23:43:34.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.675+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(17 [pgnls start_epoch 33 out=254b] v33'14 uv14 ondisk = 1) v8 ==== 144+0+254 (crc 0 0 0) 0x557792f13200 con 0x557792f4dc00 2024-09-13T23:43:34.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.675+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.11 6:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557793052000 2024-09-13T23:43:34.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.675+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(18 [pgnls start_epoch 33 out=213b] v33'19 uv11 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x557792f13200 con 0x557793052000 2024-09-13T23:43:34.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.675+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:19 6.9 6:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557793052000 2024-09-13T23:43:34.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.675+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(19 [pgnls start_epoch 33 out=172b] v33'31 uv23 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x557792f13200 con 0x557793052000 2024-09-13T23:43:34.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.675+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:20 6.19 6:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792fe6400 2024-09-13T23:43:34.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.675+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(20 [pgnls start_epoch 33 out=295b] v33'29 uv25 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x557792f13440 con 0x557792fe6400 2024-09-13T23:43:34.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.675+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.5 6:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792fe6400 2024-09-13T23:43:34.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.679+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(21 [pgnls start_epoch 33 out=276b] v33'13 uv13 ondisk = 1) v8 ==== 144+0+276 (crc 0 0 0) 0x557792f13440 con 0x557792fe6400 2024-09-13T23:43:34.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.679+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.15 6:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792fe6400 2024-09-13T23:43:34.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.679+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(22 [pgnls start_epoch 33 out=172b] v33'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x557792f13440 con 0x557792fe6400 2024-09-13T23:43:34.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.679+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:23 6.d 6:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792fe6400 2024-09-13T23:43:34.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.679+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(23 [pgnls start_epoch 33 out=295b] v33'25 uv17 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x557792f13440 con 0x557792fe6400 2024-09-13T23:43:34.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.679+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 6.1d 6:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792f4dc00 2024-09-13T23:43:34.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.679+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(24 [pgnls start_epoch 33 out=172b] v33'29 uv12 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x557792f13200 con 0x557792f4dc00 2024-09-13T23:43:34.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.679+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:25 6.3 6:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792fe6400 2024-09-13T23:43:34.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.683+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=213b] v33'43 uv19 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x557792f13440 con 0x557792fe6400 2024-09-13T23:43:34.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.683+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.13 6:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557793052000 2024-09-13T23:43:34.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.683+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(26 [pgnls start_epoch 33 out=172b] v33'19 uv11 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x557792f13200 con 0x557793052000 2024-09-13T23:43:34.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.683+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.b 6:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557793052000 2024-09-13T23:43:34.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.683+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(27 [pgnls start_epoch 33 out=172b] v33'17 uv9 ondisk = 1) v8 ==== 144+0+172 (crc 0 0 0) 0x557792f13200 con 0x557793052000 2024-09-13T23:43:34.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.683+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:28 6.1b 6:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557793052000 2024-09-13T23:43:34.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.683+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(28 [pgnls start_epoch 33 out=90b] v33'13 uv5 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x557792f13200 con 0x557793052000 2024-09-13T23:43:34.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.683+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:29 6.7 6:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792fe6400 2024-09-13T23:43:34.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.683+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(29 [pgnls start_epoch 33 out=90b] v33'37 uv13 ondisk = 1) v8 ==== 144+0+90 (crc 0 0 0) 0x557792f13440 con 0x557792fe6400 2024-09-13T23:43:34.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.683+0000 7f9e83ec5700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:30 6.17 6:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792f4dc00 2024-09-13T23:43:34.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.687+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(30 [pgnls start_epoch 33 out=131b] v33'7 uv6 ondisk = 1) v8 ==== 144+0+131 (crc 0 0 0) 0x557792f13200 con 0x557792f4dc00 2024-09-13T23:43:34.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.687+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:31 6.f 6:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557792f4c000 2024-09-13T23:43:34.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.687+0000 7f9e84ec7700 1 -- 172.21.15.167:0/3940404918 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(31 [pgnls start_epoch 33 out=295b] v33'23 uv15 ondisk = 1) v8 ==== 144+0+295 (crc 0 0 0) 0x557792f12b40 con 0x557792f4c000 2024-09-13T23:43:34.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.687+0000 7f9e84ec7700 1 -- 172.21.15.167:0/3940404918 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:32 6.1f 6:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792f4cc00 con 0x557793052000 2024-09-13T23:43:34.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.687+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(32 [pgnls start_epoch 33 out=213b] v33'19 uv11 ondisk = 1) v8 ==== 144+0+213 (crc 0 0 0) 0x557792f13200 con 0x557793052000 2024-09-13T23:43:34.697 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-09-13T23:43:34.697 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000078", 2024-09-13T23:43:34.697 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000070", 2024-09-13T23:43:34.697 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000104", 2024-09-13T23:43:34.697 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000026", 2024-09-13T23:43:34.697 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000028", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000040", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000015", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000069", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000095", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000003", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000047", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000035", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000037", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000024", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000000", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000031", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "data_loggenerations_metadata", 2024-09-13T23:43:34.698 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000076", 2024-09-13T23:43:34.699 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000113", 2024-09-13T23:43:34.699 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000005", 2024-09-13T23:43:34.699 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000011", 2024-09-13T23:43:34.699 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000058", 2024-09-13T23:43:34.699 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000052", 2024-09-13T23:43:34.699 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000088", 2024-09-13T23:43:34.699 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000080", 2024-09-13T23:43:34.699 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000090", 2024-09-13T23:43:34.700 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000110", 2024-09-13T23:43:34.700 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000096", 2024-09-13T23:43:34.700 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000087", 2024-09-13T23:43:34.700 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000008", 2024-09-13T23:43:34.700 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000006", 2024-09-13T23:43:34.700 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000029", 2024-09-13T23:43:34.700 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000089", 2024-09-13T23:43:34.700 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000022", 2024-09-13T23:43:34.700 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000016", 2024-09-13T23:43:34.700 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000041", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000018", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000092", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000014", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000112", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000007", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000021", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000064", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000071", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000074", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000081", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000009", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000121", 2024-09-13T23:43:34.701 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000125", 2024-09-13T23:43:34.702 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000082", 2024-09-13T23:43:34.702 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000105", 2024-09-13T23:43:34.702 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000059", 2024-09-13T23:43:34.702 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000077", 2024-09-13T23:43:34.702 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000032", 2024-09-13T23:43:34.702 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000053", 2024-09-13T23:43:34.702 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000091", 2024-09-13T23:43:34.702 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000065", 2024-09-13T23:43:34.702 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000083", 2024-09-13T23:43:34.702 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000010", 2024-09-13T23:43:34.702 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000045", 2024-09-13T23:43:34.702 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000002", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000116", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000034", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000101", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000079", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000049", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000117", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000044", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000066", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000068", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000085", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000073", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000038", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000118", 2024-09-13T23:43:34.703 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000036", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000103", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000119", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000098", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000027", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000019", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000039", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000100", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000093", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000004", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000063", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000122", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000057", 2024-09-13T23:43:34.704 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000054", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000114", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "data_log.0", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000012", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000084", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000043", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000111", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000048", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000020", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000099", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000056", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000072", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000062", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000115", 2024-09-13T23:43:34.705 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000033", 2024-09-13T23:43:34.706 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000025", 2024-09-13T23:43:34.706 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000123", 2024-09-13T23:43:34.706 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000108", 2024-09-13T23:43:34.706 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000094", 2024-09-13T23:43:34.706 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000060", 2024-09-13T23:43:34.706 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000109", 2024-09-13T23:43:34.706 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000013", 2024-09-13T23:43:34.706 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000051", 2024-09-13T23:43:34.706 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000106", 2024-09-13T23:43:34.706 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000017", 2024-09-13T23:43:34.706 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000023", 2024-09-13T23:43:34.707 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000107", 2024-09-13T23:43:34.707 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000046", 2024-09-13T23:43:34.707 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000067", 2024-09-13T23:43:34.707 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000086", 2024-09-13T23:43:34.707 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000050", 2024-09-13T23:43:34.707 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000102", 2024-09-13T23:43:34.707 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000124", 2024-09-13T23:43:34.707 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000001", 2024-09-13T23:43:34.707 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000126", 2024-09-13T23:43:34.707 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000055", 2024-09-13T23:43:34.707 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000061", 2024-09-13T23:43:34.707 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000097", 2024-09-13T23:43:34.708 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000042", 2024-09-13T23:43:34.708 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000075", 2024-09-13T23:43:34.708 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000120", 2024-09-13T23:43:34.708 INFO:teuthology.orchestra.run.smithi167.stdout: "obj_delete_at_hint.0000000030" 2024-09-13T23:43:34.708 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-09-13T23:43:34.708 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:34.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.691+0000 7f9e6b694700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:34.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.691+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93972055040000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792fe6800 con 0x557792e85000 2024-09-13T23:43:34.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.695+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 data_loggenerations_metadata [watch unwatch cookie 93972055040000] v33'147 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.695+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93972053359360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792fe6c00 con 0x557792e85400 2024-09-13T23:43:34.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.695+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(63 notify.0 [watch unwatch cookie 93972053359360] v33'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.699+0000 7f9e85f0df40 20 remove_watcher() i=0 2024-09-13T23:43:34.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.699+0000 7f9e85f0df40 2 removed watcher, disabling cache 2024-09-13T23:43:34.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.699+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93972053360512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792fe7000 con 0x557792e85000 2024-09-13T23:43:34.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.699+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(64 notify.1 [watch unwatch cookie 93972053360512] v33'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85000 2024-09-13T23:43:34.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.699+0000 7f9e85f0df40 20 remove_watcher() i=1 2024-09-13T23:43:34.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.699+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93972053361664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792fe7400 con 0x557792e85400 2024-09-13T23:43:34.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.703+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 notify.2 [watch unwatch cookie 93972053361664] v33'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.703+0000 7f9e85f0df40 20 remove_watcher() i=2 2024-09-13T23:43:34.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.703+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93972053362816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792fe7800 con 0x557792e85c00 2024-09-13T23:43:34.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.707+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(66 notify.3 [watch unwatch cookie 93972053362816] v33'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.707+0000 7f9e85f0df40 20 remove_watcher() i=3 2024-09-13T23:43:34.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.707+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93972053363968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557792fe7c00 con 0x557792e85400 2024-09-13T23:43:34.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.707+0000 7f9e846c6700 1 -- 172.21.15.167:0/173588780 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(67 notify.4 [watch unwatch cookie 93972053363968] v33'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13200 con 0x557792e85400 2024-09-13T23:43:34.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.707+0000 7f9e85f0df40 20 remove_watcher() i=4 2024-09-13T23:43:34.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.707+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93972053365120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557793052c00 con 0x557792e85800 2024-09-13T23:43:34.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.711+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(68 notify.5 [watch unwatch cookie 93972053365120] v33'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.711+0000 7f9e85f0df40 20 remove_watcher() i=5 2024-09-13T23:43:34.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.711+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93972053366272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557793053000 con 0x557792e85800 2024-09-13T23:43:34.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(69 notify.6 [watch unwatch cookie 93972053366272] v33'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f13440 con 0x557792e85800 2024-09-13T23:43:34.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 20 remove_watcher() i=6 2024-09-13T23:43:34.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93972053367424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x557793053400 con 0x557792e85c00 2024-09-13T23:43:34.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e84ec7700 1 -- 172.21.15.167:0/173588780 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(70 notify.7 [watch unwatch cookie 93972053367424] v33'152 uv150 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x557792f12b40 con 0x557792e85c00 2024-09-13T23:43:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 20 remove_watcher() i=7 2024-09-13T23:43:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557792e85400 msgr2=0x557792dc9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557792e85400 0x557792dc9b80 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557792e85000 msgr2=0x557792dc9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557792e85000 0x557792dc9600 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557792e85c00 msgr2=0x557792dc8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557792e85c00 0x557792dc8580 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557792e85800 msgr2=0x557792dcac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557792e85800 0x557792dcac00 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792e84c00 msgr2=0x557792dca680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792e84c00 0x557792dca680 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x5577938ac030 tx=0x557792f3c0c0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 msgr2=0x557792dc9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e83ec5700 1 -- 172.21.15.167:0/173588780 reap_dead start 2024-09-13T23:43:34.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 0x557792dc9080 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x557792e0a870 tx=0x557792e465a0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 shutdown_connections 2024-09-13T23:43:34.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/173588780 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3c00 0x557792dc9080 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.715+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 >> 172.21.15.167:0/173588780 conn(0x557792ae3800 msgr2=0x557792be3800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 shutdown_connections 2024-09-13T23:43:34.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/173588780 wait complete. 2024-09-13T23:43:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557792fe6400 msgr2=0x557792fec580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x557792fe6400 0x557792fec580 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557792f4dc00 msgr2=0x557792dcb700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x557792f4dc00 0x557792dcb700 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557792f4c000 msgr2=0x557792fecb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x557792f4c000 0x557792fecb00 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557793052000 msgr2=0x557792fed080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x557793052000 0x557792fed080 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792fe6000 msgr2=0x557792fec000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792fe6000 0x557792fec000 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x557792e62330 tx=0x557792e47500 comp rx=0 tx=0).stop 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 msgr2=0x557792dcb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/3940404918 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792f4d400 0x557792dcb180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x557792f65260 tx=0x557792f3c180 comp rx=0 tx=0).stop 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 reap_dead start 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e846c6700 1 -- 172.21.15.167:0/3940404918 reap_dead start 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 shutdown_connections 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 >> 172.21.15.167:0/3940404918 conn(0x557792f4c800 msgr2=0x557792be4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 shutdown_connections 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/3940404918 wait complete. 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2843038813 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792e84000 msgr2=0x557792dca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/2843038813 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792e84000 0x557792dca100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x557792e0b620 tx=0x557792e46c60 comp rx=0 tx=0).stop 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2843038813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3400 msgr2=0x557792dc8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/2843038813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3400 0x557792dc8b00 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x5577938ade90 tx=0x557792be7920 comp rx=0 tx=0).stop 2024-09-13T23:43:34.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2843038813 shutdown_connections 2024-09-13T23:43:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/2843038813 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x557792e84000 0x557792dca100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 --2- 172.21.15.167:0/2843038813 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x557792ae3400 0x557792dc8b00 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2843038813 >> 172.21.15.167:0/2843038813 conn(0x557792ae2c00 msgr2=0x557792be3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2843038813 shutdown_connections 2024-09-13T23:43:34.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.719+0000 7f9e85f0df40 1 -- 172.21.15.167:0/2843038813 wait complete. 2024-09-13T23:43:34.736 DEBUG:tasks.util.rgw: json result: ['obj_delete_at_hint.0000000078', 'obj_delete_at_hint.0000000070', 'obj_delete_at_hint.0000000104', 'obj_delete_at_hint.0000000026', 'obj_delete_at_hint.0000000028', 'obj_delete_at_hint.0000000040', 'obj_delete_at_hint.0000000015', 'obj_delete_at_hint.0000000069', 'obj_delete_at_hint.0000000095', 'obj_delete_at_hint.0000000003', 'obj_delete_at_hint.0000000047', 'obj_delete_at_hint.0000000035', 'obj_delete_at_hint.0000000037', 'obj_delete_at_hint.0000000024', 'obj_delete_at_hint.0000000000', 'obj_delete_at_hint.0000000031', 'data_loggenerations_metadata', 'obj_delete_at_hint.0000000076', 'obj_delete_at_hint.0000000113', 'obj_delete_at_hint.0000000005', 'obj_delete_at_hint.0000000011', 'obj_delete_at_hint.0000000058', 'obj_delete_at_hint.0000000052', 'obj_delete_at_hint.0000000088', 'obj_delete_at_hint.0000000080', 'obj_delete_at_hint.0000000090', 'obj_delete_at_hint.0000000110', 'obj_delete_at_hint.0000000096', 'obj_delete_at_hint.0000000087', 'obj_delete_at_hint.0000000008', 'obj_delete_at_hint.0000000006', 'obj_delete_at_hint.0000000029', 'obj_delete_at_hint.0000000089', 'obj_delete_at_hint.0000000022', 'obj_delete_at_hint.0000000016', 'obj_delete_at_hint.0000000041', 'obj_delete_at_hint.0000000018', 'obj_delete_at_hint.0000000092', 'obj_delete_at_hint.0000000014', 'obj_delete_at_hint.0000000112', 'obj_delete_at_hint.0000000007', 'obj_delete_at_hint.0000000021', 'obj_delete_at_hint.0000000064', 'obj_delete_at_hint.0000000071', 'obj_delete_at_hint.0000000074', 'obj_delete_at_hint.0000000081', 'obj_delete_at_hint.0000000009', 'obj_delete_at_hint.0000000121', 'obj_delete_at_hint.0000000125', 'obj_delete_at_hint.0000000082', 'obj_delete_at_hint.0000000105', 'obj_delete_at_hint.0000000059', 'obj_delete_at_hint.0000000077', 'obj_delete_at_hint.0000000032', 'obj_delete_at_hint.0000000053', 'obj_delete_at_hint.0000000091', 'obj_delete_at_hint.0000000065', 'obj_delete_at_hint.0000000083', 'obj_delete_at_hint.0000000010', 'obj_delete_at_hint.0000000045', 'obj_delete_at_hint.0000000002', 'obj_delete_at_hint.0000000116', 'obj_delete_at_hint.0000000034', 'obj_delete_at_hint.0000000101', 'obj_delete_at_hint.0000000079', 'obj_delete_at_hint.0000000049', 'obj_delete_at_hint.0000000117', 'obj_delete_at_hint.0000000044', 'obj_delete_at_hint.0000000066', 'obj_delete_at_hint.0000000068', 'obj_delete_at_hint.0000000085', 'obj_delete_at_hint.0000000073', 'obj_delete_at_hint.0000000038', 'obj_delete_at_hint.0000000118', 'obj_delete_at_hint.0000000036', 'obj_delete_at_hint.0000000103', 'obj_delete_at_hint.0000000119', 'obj_delete_at_hint.0000000098', 'obj_delete_at_hint.0000000027', 'obj_delete_at_hint.0000000019', 'obj_delete_at_hint.0000000039', 'obj_delete_at_hint.0000000100', 'obj_delete_at_hint.0000000093', 'obj_delete_at_hint.0000000004', 'obj_delete_at_hint.0000000063', 'obj_delete_at_hint.0000000122', 'obj_delete_at_hint.0000000057', 'obj_delete_at_hint.0000000054', 'obj_delete_at_hint.0000000114', 'data_log.0', 'obj_delete_at_hint.0000000012', 'obj_delete_at_hint.0000000084', 'obj_delete_at_hint.0000000043', 'obj_delete_at_hint.0000000111', 'obj_delete_at_hint.0000000048', 'obj_delete_at_hint.0000000020', 'obj_delete_at_hint.0000000099', 'obj_delete_at_hint.0000000056', 'obj_delete_at_hint.0000000072', 'obj_delete_at_hint.0000000062', 'obj_delete_at_hint.0000000115', 'obj_delete_at_hint.0000000033', 'obj_delete_at_hint.0000000025', 'obj_delete_at_hint.0000000123', 'obj_delete_at_hint.0000000108', 'obj_delete_at_hint.0000000094', 'obj_delete_at_hint.0000000060', 'obj_delete_at_hint.0000000109', 'obj_delete_at_hint.0000000013', 'obj_delete_at_hint.0000000051', 'obj_delete_at_hint.0000000106', 'obj_delete_at_hint.0000000017', 'obj_delete_at_hint.0000000023', 'obj_delete_at_hint.0000000107', 'obj_delete_at_hint.0000000046', 'obj_delete_at_hint.0000000067', 'obj_delete_at_hint.0000000086', 'obj_delete_at_hint.0000000050', 'obj_delete_at_hint.0000000102', 'obj_delete_at_hint.0000000124', 'obj_delete_at_hint.0000000001', 'obj_delete_at_hint.0000000126', 'obj_delete_at_hint.0000000055', 'obj_delete_at_hint.0000000061', 'obj_delete_at_hint.0000000097', 'obj_delete_at_hint.0000000042', 'obj_delete_at_hint.0000000075', 'obj_delete_at_hint.0000000120', 'obj_delete_at_hint.0000000030'] 2024-09-13T23:43:34.736 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-09-13T23:43:34.736 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-09-13T23:43:34.736 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-09-13T23:43:34.779 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:34.779 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:34.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aceedf40 1 Processor -- start 2024-09-13T23:43:34.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aceedf40 1 -- start start 2024-09-13T23:43:34.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aceedf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5619c815b000 0x5619c8440580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aceedf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5619c821f500 con 0x5619c815b400 2024-09-13T23:43:34.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aceedf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5619c81e8a00 con 0x5619c815b000 2024-09-13T23:43:34.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91ab6a6700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5619c815b000 0x5619c8440580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aaea5700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5619c815b400 0x5619c825a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45574/0 (socket says 172.21.15.167:45574) 2024-09-13T23:43:34.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aaea5700 1 -- 172.21.15.167:0/1655425207 learned_addr learned my addr 172.21.15.167:0/1655425207 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:34.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aa6a4700 1 -- 172.21.15.167:0/1655425207 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079151122 0 0) 0x5619c82b6000 con 0x5619c815b400 2024-09-13T23:43:34.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aa6a4700 1 -- 172.21.15.167:0/1655425207 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x5619c821f680 con 0x5619c815b400 2024-09-13T23:43:34.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91ab6a6700 1 -- 172.21.15.167:0/1655425207 >> v1:172.21.15.12:6789/0 conn(0x5619c815b400 legacy=0x5619c825a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91ab6a6700 1 -- 172.21.15.167:0/1655425207 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c816dd40 con 0x5619c815b000 2024-09-13T23:43:34.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91ab6a6700 1 --2- 172.21.15.167:0/1655425207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x5619c843ba70 tx=0x5619c825f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea2059903b4b35d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aa6a4700 1 -- 172.21.15.167:0/1655425207 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c81ff4a0 con 0x5619c815b000 2024-09-13T23:43:34.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aa6a4700 1 -- 172.21.15.167:0/1655425207 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619c821f680 con 0x5619c815b000 2024-09-13T23:43:34.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aa6a4700 1 -- 172.21.15.167:0/1655425207 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c81ff600 con 0x5619c815b000 2024-09-13T23:43:34.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aceedf40 1 -- 172.21.15.167:0/1655425207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 msgr2=0x5619c8440580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.771+0000 7f91aceedf40 1 --2- 172.21.15.167:0/1655425207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x5619c843ba70 tx=0x5619c825f980 comp rx=0 tx=0).stop 2024-09-13T23:43:34.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91aceedf40 1 -- 172.21.15.167:0/1655425207 shutdown_connections 2024-09-13T23:43:34.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91aceedf40 1 --2- 172.21.15.167:0/1655425207 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91aceedf40 1 -- 172.21.15.167:0/1655425207 >> 172.21.15.167:0/1655425207 conn(0x5619c815ac00 msgr2=0x5619c825a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91aceedf40 1 -- 172.21.15.167:0/1655425207 shutdown_connections 2024-09-13T23:43:34.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91aceedf40 1 -- 172.21.15.167:0/1655425207 wait complete. 2024-09-13T23:43:34.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91aceedf40 1 Processor -- start 2024-09-13T23:43:34.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91aceedf40 1 -- start start 2024-09-13T23:43:34.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91aceedf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91aceedf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c81e8500 con 0x5619c815b000 2024-09-13T23:43:34.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91ab6a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91ab6a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53086/0 (socket says 172.21.15.167:53086) 2024-09-13T23:43:34.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91ab6a6700 1 -- 172.21.15.167:0/272704589 learned_addr learned my addr 172.21.15.167:0/272704589 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91ab6a6700 1 -- 172.21.15.167:0/272704589 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c816dba0 con 0x5619c815b000 2024-09-13T23:43:34.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.775+0000 7f91ab6a6700 1 --2- 172.21.15.167:0/272704589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x5619c8f25e00 tx=0x5619c825f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ba05bf62b7113b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91a9ea3700 1 -- 172.21.15.167:0/272704589 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c81ff760 con 0x5619c815b000 2024-09-13T23:43:34.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91a9ea3700 1 -- 172.21.15.167:0/272704589 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619c821f980 con 0x5619c815b000 2024-09-13T23:43:34.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91a9ea3700 1 -- 172.21.15.167:0/272704589 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c81ff4a0 con 0x5619c815b000 2024-09-13T23:43:34.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91aceedf40 1 -- 172.21.15.167:0/272704589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 msgr2=0x5619c8440580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91aceedf40 1 --2- 172.21.15.167:0/272704589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x5619c8f25e00 tx=0x5619c825f800 comp rx=0 tx=0).stop 2024-09-13T23:43:34.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91aceedf40 1 -- 172.21.15.167:0/272704589 shutdown_connections 2024-09-13T23:43:34.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91aceedf40 1 --2- 172.21.15.167:0/272704589 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91aceedf40 1 -- 172.21.15.167:0/272704589 >> 172.21.15.167:0/272704589 conn(0x5619c815ac00 msgr2=0x5619c825b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91aceedf40 1 -- 172.21.15.167:0/272704589 shutdown_connections 2024-09-13T23:43:34.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91aceedf40 1 -- 172.21.15.167:0/272704589 wait complete. 2024-09-13T23:43:34.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91aceedf40 1 Processor -- start 2024-09-13T23:43:34.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91aceedf40 1 -- start start 2024-09-13T23:43:34.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91aceedf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91aceedf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c81e83c0 con 0x5619c815b000 2024-09-13T23:43:34.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91ab6a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91ab6a6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53102/0 (socket says 172.21.15.167:53102) 2024-09-13T23:43:34.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.779+0000 7f91ab6a6700 1 -- 172.21.15.167:0/2699246509 learned_addr learned my addr 172.21.15.167:0/2699246509 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91ab6a6700 1 -- 172.21.15.167:0/2699246509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c816da00 con 0x5619c815b000 2024-09-13T23:43:34.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91ab6a6700 1 --2- 172.21.15.167:0/2699246509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x5619c843bf80 tx=0x5619c825f980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91a86a0700 1 -- 172.21.15.167:0/2699246509 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c81ff600 con 0x5619c815b000 2024-09-13T23:43:34.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91a86a0700 1 -- 172.21.15.167:0/2699246509 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619c821fb00 con 0x5619c815b000 2024-09-13T23:43:34.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91aceedf40 1 -- 172.21.15.167:0/2699246509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619c816d860 con 0x5619c815b000 2024-09-13T23:43:34.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91a86a0700 1 -- 172.21.15.167:0/2699246509 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c81ff8c0 con 0x5619c815b000 2024-09-13T23:43:34.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91aceedf40 1 -- 172.21.15.167:0/2699246509 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619c816d6c0 con 0x5619c815b000 2024-09-13T23:43:34.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91aceedf40 1 Processor -- start 2024-09-13T23:43:34.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91aceedf40 1 -- start start 2024-09-13T23:43:34.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91aceedf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 0x5619c8440b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91a86a0700 1 -- 172.21.15.167:0/2699246509 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5619c8254b00 con 0x5619c815b000 2024-09-13T23:43:34.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91aceedf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c8180640 con 0x5619c815b800 2024-09-13T23:43:34.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91abea7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 0x5619c8440b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.783+0000 7f91abea7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 0x5619c8440b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53118/0 (socket says 172.21.15.167:53118) 2024-09-13T23:43:34.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91abea7700 1 -- 172.21.15.167:0/206309305 learned_addr learned my addr 172.21.15.167:0/206309305 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91a86a0700 1 --2- 172.21.15.167:0/2699246509 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8504000 0x5619c8441080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91a86a0700 1 -- 172.21.15.167:0/2699246509 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x5619c81c1180 con 0x5619c815b000 2024-09-13T23:43:34.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91ab6a6700 1 --2- 172.21.15.167:0/2699246509 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8504000 0x5619c8441080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91abea7700 1 -- 172.21.15.167:0/206309305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c816c000 con 0x5619c815b800 2024-09-13T23:43:34.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91abea7700 1 --2- 172.21.15.167:0/206309305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 0x5619c8440b00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5619c84e4510 tx=0x5619c84c2b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236dc47969455d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91ab6a6700 1 --2- 172.21.15.167:0/2699246509 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8504000 0x5619c8441080 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x5619c84e4570 tx=0x5619c84c2c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91a5e9b700 1 -- 172.21.15.167:0/206309305 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c81ff600 con 0x5619c815b800 2024-09-13T23:43:34.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91a5e9b700 1 -- 172.21.15.167:0/206309305 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619c821fb00 con 0x5619c815b800 2024-09-13T23:43:34.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91a5e9b700 1 -- 172.21.15.167:0/206309305 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c81ff8c0 con 0x5619c815b800 2024-09-13T23:43:34.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91aceedf40 1 -- 172.21.15.167:0/206309305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 msgr2=0x5619c8440b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91aceedf40 1 --2- 172.21.15.167:0/206309305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 0x5619c8440b00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x5619c84e4510 tx=0x5619c84c2b40 comp rx=0 tx=0).stop 2024-09-13T23:43:34.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91aceedf40 1 -- 172.21.15.167:0/206309305 shutdown_connections 2024-09-13T23:43:34.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91aceedf40 1 --2- 172.21.15.167:0/206309305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 0x5619c8440b00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91aceedf40 1 -- 172.21.15.167:0/206309305 >> 172.21.15.167:0/206309305 conn(0x5619c815b400 msgr2=0x5619c825b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91aceedf40 1 -- 172.21.15.167:0/206309305 shutdown_connections 2024-09-13T23:43:34.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91aceedf40 1 -- 172.21.15.167:0/206309305 wait complete. 2024-09-13T23:43:34.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91aceedf40 1 Processor -- start 2024-09-13T23:43:34.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91aceedf40 1 -- start start 2024-09-13T23:43:34.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91aceedf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 0x5619c8440b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91aceedf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c81808c0 con 0x5619c815b800 2024-09-13T23:43:34.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91abea7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 0x5619c8440b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91abea7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 0x5619c8440b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53124/0 (socket says 172.21.15.167:53124) 2024-09-13T23:43:34.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.787+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 learned_addr learned my addr 172.21.15.167:0/114867172 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c816cea0 con 0x5619c815b800 2024-09-13T23:43:34.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91abea7700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 0x5619c8440b00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5619c8fd2930 tx=0x5619c84c25a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91a4698700 1 -- 172.21.15.167:0/114867172 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c81ff760 con 0x5619c815b800 2024-09-13T23:43:34.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91a4698700 1 -- 172.21.15.167:0/114867172 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619c8528000 con 0x5619c815b800 2024-09-13T23:43:34.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619c816c820 con 0x5619c815b800 2024-09-13T23:43:34.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91a4698700 1 -- 172.21.15.167:0/114867172 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c81ff600 con 0x5619c815b800 2024-09-13T23:43:34.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619c816c9c0 con 0x5619c815b800 2024-09-13T23:43:34.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91a4698700 1 -- 172.21.15.167:0/114867172 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5619c8255080 con 0x5619c815b800 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91a4698700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8504c00 0x5619c8441600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91a4698700 1 -- 172.21.15.167:0/114867172 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x5619c81c1340 con 0x5619c815b800 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91aaea5700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8504c00 0x5619c8441600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91aceedf40 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619c8505000 0x5619c8441b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.791+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505400 con 0x5619c8505000 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91abea7700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619c8505000 0x5619c8441b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aaea5700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8504c00 0x5619c8441600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x5619c8f24030 tx=0x5619c85b40c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91abea7700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619c8505000 0x5619c8441b80 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aceedf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aceedf40 20 realm 2024-09-13T23:43:34.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505000 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aceedf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aceedf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505c00 con 0x5619c8505000 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aceedf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aceedf40 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619c8505400 0x5619c8442100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505400 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91ab6a6700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619c8505400 0x5619c8442100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.795+0000 7f91ab6a6700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619c8505400 0x5619c8442100 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91aceedf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:34.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505c00 con 0x5619c8505400 2024-09-13T23:43:34.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91aceedf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:34.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91aceedf40 20 searching for the correct realm 2024-09-13T23:43:34.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91aceedf40 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619c85c0000 0x5619c8442680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c0400 con 0x5619c85c0000 2024-09-13T23:43:34.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91aaea5700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619c85c0000 0x5619c8442680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91aaea5700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619c85c0000 0x5619c8442680 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c0400 con 0x5619c8505400 2024-09-13T23:43:34.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91ab6a6700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619c85c0400 0x5619c8442c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.799+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0400 2024-09-13T23:43:34.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.803+0000 7f91abea7700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619c85c0400 0x5619c8442c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.803+0000 7f91abea7700 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619c85c0400 0x5619c8442c00 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.803+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.803+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505000 2024-09-13T23:43:34.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.803+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.803+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505000 2024-09-13T23:43:34.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.803+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.803+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0000 2024-09-13T23:43:34.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.803+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.803+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505000 2024-09-13T23:43:34.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.803+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.803+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505400 2024-09-13T23:43:34.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505400 2024-09-13T23:43:34.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505000 2024-09-13T23:43:34.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0400 2024-09-13T23:43:34.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505000 2024-09-13T23:43:34.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0400 2024-09-13T23:43:34.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505400 2024-09-13T23:43:34.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.807+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0000 2024-09-13T23:43:34.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505000 2024-09-13T23:43:34.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505400 2024-09-13T23:43:34.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0000 2024-09-13T23:43:34.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505000 2024-09-13T23:43:34.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505000 2024-09-13T23:43:34.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.811+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505000 2024-09-13T23:43:34.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0000 2024-09-13T23:43:34.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0400 2024-09-13T23:43:34.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505000 2024-09-13T23:43:34.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0400 2024-09-13T23:43:34.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0000 2024-09-13T23:43:34.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0400 2024-09-13T23:43:34.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.815+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505400 2024-09-13T23:43:34.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505400 2024-09-13T23:43:34.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0000 2024-09-13T23:43:34.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505400 2024-09-13T23:43:34.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c8505400 2024-09-13T23:43:34.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91aceedf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:34.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91aceedf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:34.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91aceedf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:34.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91aceedf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:34.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.819+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c0800 con 0x5619c8505000 2024-09-13T23:43:34.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c0c00 con 0x5619c8505400 2024-09-13T23:43:34.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:34.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c1000 con 0x5619c8505400 2024-09-13T23:43:34.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:34.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 20 zone default found 2024-09-13T23:43:34.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 4 Realm: () 2024-09-13T23:43:34.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:34.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:34.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:34.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 20 zonegroup default 2024-09-13T23:43:34.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c1400 con 0x5619c85c0400 2024-09-13T23:43:34.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.823+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c1800 con 0x5619c85c0000 2024-09-13T23:43:34.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.827+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.827+0000 7f91aceedf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.827+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.827+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c1c00 con 0x5619c85c0000 2024-09-13T23:43:34.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.827+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.827+0000 7f91aceedf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.827+0000 7f91aceedf40 20 started sync module instance, tier type = 2024-09-13T23:43:34.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.827+0000 7f91aceedf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:34.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.827+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c1800 con 0x5619c8505400 2024-09-13T23:43:34.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.831+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.831+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94668734036736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c1c00 con 0x5619c8505400 2024-09-13T23:43:34.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.831+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c0c00 con 0x5619c8505000 2024-09-13T23:43:34.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.831+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.831+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94668734036736] v33'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.831+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94668734037888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c1000 con 0x5619c8505000 2024-09-13T23:43:34.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.831+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c1800 con 0x5619c8505400 2024-09-13T23:43:34.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.835+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94668734037888] v33'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.835+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.835+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94668734039040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505c00 con 0x5619c8505400 2024-09-13T23:43:34.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.835+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0400 2024-09-13T23:43:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.839+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.839+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94668734040192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c0800 con 0x5619c85c0400 2024-09-13T23:43:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.839+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c1400 con 0x5619c8505400 2024-09-13T23:43:34.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.839+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94668734039040] v33'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.839+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94668734040192] v33'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.839+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.839+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94668734041344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c0c00 con 0x5619c8505400 2024-09-13T23:43:34.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.839+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c1000 con 0x5619c85c0000 2024-09-13T23:43:34.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.843+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.843+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94668734042496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85ca000 con 0x5619c85c0000 2024-09-13T23:43:34.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.843+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85ca400 con 0x5619c85c0000 2024-09-13T23:43:34.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.843+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94668734041344] v33'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.843+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94668734042496] v33'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.843+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v33'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.843+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94668734043648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85ca800 con 0x5619c85c0000 2024-09-13T23:43:34.854 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.843+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85cac00 con 0x5619c85c0400 2024-09-13T23:43:34.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.847+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94668734043648] v33'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.847+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'153 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.847+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94668734044800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85cb000 con 0x5619c85c0400 2024-09-13T23:43:34.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.847+0000 7f91aceedf40 20 add_watcher() i=0 2024-09-13T23:43:34.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.847+0000 7f91aceedf40 20 add_watcher() i=1 2024-09-13T23:43:34.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.847+0000 7f91aceedf40 20 add_watcher() i=2 2024-09-13T23:43:34.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.847+0000 7f91aceedf40 20 add_watcher() i=3 2024-09-13T23:43:34.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.847+0000 7f91aceedf40 20 add_watcher() i=4 2024-09-13T23:43:34.857 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.847+0000 7f91aceedf40 20 add_watcher() i=5 2024-09-13T23:43:34.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.847+0000 7f91aceedf40 20 add_watcher() i=6 2024-09-13T23:43:34.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94668734044800] v33'154 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91aceedf40 20 add_watcher() i=7 2024-09-13T23:43:34.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91aceedf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:34.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85cb400 con 0x5619c8505000 2024-09-13T23:43:34.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94668735709184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85cb800 con 0x5619c8505000 2024-09-13T23:43:34.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94668735709184] v33'148 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f9192674700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:34.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91aceedf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:34.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91aceedf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:34.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91aceedf40 1 Processor -- start 2024-09-13T23:43:34.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91aceedf40 1 -- start start 2024-09-13T23:43:34.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91aceedf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 0x5619c8443180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.851+0000 7f91aceedf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c81e83c0 con 0x5619c85cac00 2024-09-13T23:43:34.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aaea5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 0x5619c8443180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aaea5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 0x5619c8443180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53128/0 (socket says 172.21.15.167:53128) 2024-09-13T23:43:34.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aaea5700 1 -- 172.21.15.167:0/1520943276 learned_addr learned my addr 172.21.15.167:0/1520943276 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aaea5700 1 -- 172.21.15.167:0/1520943276 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c816cb60 con 0x5619c85cac00 2024-09-13T23:43:34.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aaea5700 1 --2- 172.21.15.167:0/1520943276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 0x5619c8443180 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x5619c85d80c0 tx=0x5619c85b4060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9ba6642b265535 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f9191e73700 1 -- 172.21.15.167:0/1520943276 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c85dc000 con 0x5619c85cac00 2024-09-13T23:43:34.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f9191e73700 1 -- 172.21.15.167:0/1520943276 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619c8528480 con 0x5619c85cac00 2024-09-13T23:43:34.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f9191e73700 1 -- 172.21.15.167:0/1520943276 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c85dc160 con 0x5619c85cac00 2024-09-13T23:43:34.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aceedf40 1 -- 172.21.15.167:0/1520943276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 msgr2=0x5619c8443180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aceedf40 1 --2- 172.21.15.167:0/1520943276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 0x5619c8443180 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x5619c85d80c0 tx=0x5619c85b4060 comp rx=0 tx=0).stop 2024-09-13T23:43:34.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aceedf40 1 -- 172.21.15.167:0/1520943276 shutdown_connections 2024-09-13T23:43:34.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aceedf40 1 --2- 172.21.15.167:0/1520943276 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 0x5619c8443180 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aceedf40 1 -- 172.21.15.167:0/1520943276 >> 172.21.15.167:0/1520943276 conn(0x5619c85cbc00 msgr2=0x5619c825c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aceedf40 1 -- 172.21.15.167:0/1520943276 shutdown_connections 2024-09-13T23:43:34.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aceedf40 1 -- 172.21.15.167:0/1520943276 wait complete. 2024-09-13T23:43:34.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aceedf40 1 Processor -- start 2024-09-13T23:43:34.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aceedf40 1 -- start start 2024-09-13T23:43:34.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aceedf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 0x5619c8443180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aceedf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619c81e8500 con 0x5619c85cac00 2024-09-13T23:43:34.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aaea5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 0x5619c8443180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aaea5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 0x5619c8443180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53140/0 (socket says 172.21.15.167:53140) 2024-09-13T23:43:34.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.855+0000 7f91aaea5700 1 -- 172.21.15.167:0/1173162010 learned_addr learned my addr 172.21.15.167:0/1173162010 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f91aaea5700 1 -- 172.21.15.167:0/1173162010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619c816cd00 con 0x5619c85cac00 2024-09-13T23:43:34.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f91aaea5700 1 --2- 172.21.15.167:0/1173162010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 0x5619c8443180 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x5619c85d9380 tx=0x5619c85b4180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f9190670700 1 -- 172.21.15.167:0/1173162010 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c85dc2c0 con 0x5619c85cac00 2024-09-13T23:43:34.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f91aceedf40 1 -- 172.21.15.167:0/1173162010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5619c816c340 con 0x5619c85cac00 2024-09-13T23:43:34.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f91aceedf40 1 -- 172.21.15.167:0/1173162010 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5619c85e6000 con 0x5619c85cac00 2024-09-13T23:43:34.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f9190670700 1 -- 172.21.15.167:0/1173162010 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5619c8528600 con 0x5619c85cac00 2024-09-13T23:43:34.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f9190670700 1 -- 172.21.15.167:0/1173162010 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5619c85dc000 con 0x5619c85cac00 2024-09-13T23:43:34.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f9190670700 1 -- 172.21.15.167:0/1173162010 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5619c8255b80 con 0x5619c85cac00 2024-09-13T23:43:34.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f9190670700 1 --2- 172.21.15.167:0/1173162010 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8656000 0x5619c865c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f9190670700 1 -- 172.21.15.167:0/1173162010 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x5619c81c1500 con 0x5619c85cac00 2024-09-13T23:43:34.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f91aceedf40 5 note: GC not initialized 2024-09-13T23:43:34.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f91abea7700 1 --2- 172.21.15.167:0/1173162010 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8656000 0x5619c865c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.859+0000 7f91abea7700 1 --2- 172.21.15.167:0/1173162010 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8656000 0x5619c865c000 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x5619c856b800 tx=0x5619c84c3500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f918de6b700 20 reqs_thread_entry: start 2024-09-13T23:43:34.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:34.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:34.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f918be67700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:05 2024 2024-09-13T23:43:34.872 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:34.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f918b645700 20 reqs_thread_entry: start 2024-09-13T23:43:34.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:34.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85ca000 con 0x5619c8505000 2024-09-13T23:43:34.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv11 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:34.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:34.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:34.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:34.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85ca400 con 0x5619c85c0400 2024-09-13T23:43:34.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 20 disabling bucket name=myfoo 2024-09-13T23:43:34.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-13T23:43:34.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c0800 con 0x5619c85c0400 2024-09-13T23:43:34.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+165 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:34.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-09-13T23:43:34.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:34.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 : miss 2024-09-13T23:43:34.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.863+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c1000 con 0x5619c85c0000 2024-09-13T23:43:34.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(65 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aceedf40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:34.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aceedf40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:34.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aceedf40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:34.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aceedf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:34.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aceedf40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:34.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aceedf40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 2024-09-13T23:43:34.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:66 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8656400 con 0x5619c85c0000 2024-09-13T23:43:34.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(66 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v33'6 uv6 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aceedf40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:34.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aceedf40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:34.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aceedf40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:34.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aceedf40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1, ofs0, ns] 2024-09-13T23:43:34.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.867+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94668735711488 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8656c00 con 0x5619c8505400 2024-09-13T23:43:34.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94668734036736 notify 141733920797 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x5619c84b6680 con 0x5619c8505400 2024-09-13T23:43:34.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91a5e9b700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920797 cookie 94668734036736 notifier 4638 bl.length()=725 2024-09-13T23:43:34.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91a5e9b700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:34.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91a5e9b700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:34.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91a5e9b700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:34.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.0 [notify cookie 94668735711488 out=8b] v0'0 uv153 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91a5e9b700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c1c00 con 0x5619c8505400 2024-09-13T23:43:34.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94668735711488 notify 141733920797 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5619c84b6820 con 0x5619c8505400 2024-09-13T23:43:34.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91aceedf40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): orig_sources= new_sources= 2024-09-13T23:43:34.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91aceedf40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): potential sources added=[] removed=[] 2024-09-13T23:43:34.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91aceedf40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): orig_dests= new_dests= 2024-09-13T23:43:34.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91aceedf40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): potential dests added=[] removed=[] 2024-09-13T23:43:34.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.871+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8657400 con 0x5619c8505000 2024-09-13T23:43:34.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~357] v33'12 uv12 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91aceedf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:34.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91aceedf40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:34.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91aceedf40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:34.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94668735711488 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8657800 con 0x5619c8505400 2024-09-13T23:43:34.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== watch-notify(notify (1) cookie 94668734041344 notify 141733920798 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x5619c84b6820 con 0x5619c8505400 2024-09-13T23:43:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91a569a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920798 cookie 94668734041344 notifier 4638 bl.length()=533 2024-09-13T23:43:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(70 notify.4 [notify cookie 94668735711488 out=8b] v0'0 uv153 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91a569a700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91a569a700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91a569a700 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85cb800 con 0x5619c8505400 2024-09-13T23:43:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== watch-notify(notify_complete (2) cookie 94668735711488 notify 141733920798 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5619c84b69c0 con 0x5619c8505400 2024-09-13T23:43:34.885 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:34.885 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:34.885 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:34.885 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:34.885 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 1, 2024-09-13T23:43:34.885 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:34.885 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:34.885 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:34.885 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:34.885 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:34.886 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:34.887 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:34.888 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:34.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f9192674700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:34.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.875+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94668735709184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8657c00 con 0x5619c8505000 2024-09-13T23:43:34.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.879+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94668735709184] v33'149 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.879+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94668734036736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8657400 con 0x5619c8505400 2024-09-13T23:43:34.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.883+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94668734036736] v33'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.883+0000 7f91aceedf40 20 remove_watcher() i=0 2024-09-13T23:43:34.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.883+0000 7f91aceedf40 2 removed watcher, disabling cache 2024-09-13T23:43:34.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.883+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94668734037888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8657000 con 0x5619c8505000 2024-09-13T23:43:34.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.883+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94668734037888] v33'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c8505000 2024-09-13T23:43:34.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.883+0000 7f91aceedf40 20 remove_watcher() i=1 2024-09-13T23:43:34.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.883+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94668734039040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8656c00 con 0x5619c8505400 2024-09-13T23:43:34.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.887+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94668734039040] v33'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.887+0000 7f91aceedf40 20 remove_watcher() i=2 2024-09-13T23:43:34.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.887+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94668734040192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8656800 con 0x5619c85c0400 2024-09-13T23:43:34.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.891+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94668734040192] v33'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.891+0000 7f91aceedf40 20 remove_watcher() i=3 2024-09-13T23:43:34.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.891+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94668734041344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8656400 con 0x5619c8505400 2024-09-13T23:43:34.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.891+0000 7f91ab6a6700 1 -- 172.21.15.167:0/114867172 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94668734041344] v33'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f200 con 0x5619c8505400 2024-09-13T23:43:34.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.891+0000 7f91aceedf40 20 remove_watcher() i=4 2024-09-13T23:43:34.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.891+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94668734042496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505c00 con 0x5619c85c0000 2024-09-13T23:43:34.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.895+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94668734042496] v33'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.895+0000 7f91aceedf40 20 remove_watcher() i=5 2024-09-13T23:43:34.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.895+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94668734043648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c8505800 con 0x5619c85c0000 2024-09-13T23:43:34.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94668734043648] v33'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855f440 con 0x5619c85c0000 2024-09-13T23:43:34.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 20 remove_watcher() i=6 2024-09-13T23:43:34.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94668734044800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x5619c85c0800 con 0x5619c85c0400 2024-09-13T23:43:34.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91abea7700 1 -- 172.21.15.167:0/114867172 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94668734044800] v33'155 uv153 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5619c855eb40 con 0x5619c85c0400 2024-09-13T23:43:34.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 20 remove_watcher() i=7 2024-09-13T23:43:34.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619c8505400 msgr2=0x5619c8442100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5619c8505400 0x5619c8442100 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619c8505000 msgr2=0x5619c8441b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5619c8505000 0x5619c8441b80 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619c85c0400 msgr2=0x5619c8442c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5619c85c0400 0x5619c8442c00 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619c85c0000 msgr2=0x5619c8442680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5619c85c0000 0x5619c8442680 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8504c00 msgr2=0x5619c8441600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8504c00 0x5619c8441600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x5619c8f24030 tx=0x5619c85b40c0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 msgr2=0x5619c8440b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 --2- 172.21.15.167:0/114867172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b800 0x5619c8440b00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5619c8fd2930 tx=0x5619c84c25a0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 reap_dead start 2024-09-13T23:43:34.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aaea5700 1 -- 172.21.15.167:0/114867172 reap_dead start 2024-09-13T23:43:34.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 shutdown_connections 2024-09-13T23:43:34.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 >> 172.21.15.167:0/114867172 conn(0x5619c815b400 msgr2=0x5619c825b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 shutdown_connections 2024-09-13T23:43:34.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.899+0000 7f91aceedf40 1 -- 172.21.15.167:0/114867172 wait complete. 2024-09-13T23:43:34.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 -- 172.21.15.167:0/1173162010 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8656000 msgr2=0x5619c865c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 --2- 172.21.15.167:0/1173162010 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8656000 0x5619c865c000 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x5619c856b800 tx=0x5619c84c3500 comp rx=0 tx=0).stop 2024-09-13T23:43:34.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 -- 172.21.15.167:0/1173162010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 msgr2=0x5619c8443180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 --2- 172.21.15.167:0/1173162010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 0x5619c8443180 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x5619c85d9380 tx=0x5619c85b4180 comp rx=0 tx=0).stop 2024-09-13T23:43:34.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 -- 172.21.15.167:0/1173162010 shutdown_connections 2024-09-13T23:43:34.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 --2- 172.21.15.167:0/1173162010 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8656000 0x5619c865c000 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 --2- 172.21.15.167:0/1173162010 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c85cac00 0x5619c8443180 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 -- 172.21.15.167:0/1173162010 >> 172.21.15.167:0/1173162010 conn(0x5619c85cbc00 msgr2=0x5619c825c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 -- 172.21.15.167:0/1173162010 shutdown_connections 2024-09-13T23:43:34.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 -- 172.21.15.167:0/1173162010 wait complete. 2024-09-13T23:43:34.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 -- 172.21.15.167:0/2699246509 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8504000 msgr2=0x5619c8441080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 --2- 172.21.15.167:0/2699246509 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8504000 0x5619c8441080 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x5619c84e4570 tx=0x5619c84c2c60 comp rx=0 tx=0).stop 2024-09-13T23:43:34.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 -- 172.21.15.167:0/2699246509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 msgr2=0x5619c8440580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 --2- 172.21.15.167:0/2699246509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x5619c843bf80 tx=0x5619c825f980 comp rx=0 tx=0).stop 2024-09-13T23:43:34.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 -- 172.21.15.167:0/2699246509 shutdown_connections 2024-09-13T23:43:34.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 --2- 172.21.15.167:0/2699246509 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5619c8504000 0x5619c8441080 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 --2- 172.21.15.167:0/2699246509 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5619c815b000 0x5619c8440580 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 -- 172.21.15.167:0/2699246509 >> 172.21.15.167:0/2699246509 conn(0x5619c815ac00 msgr2=0x5619c825b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 -- 172.21.15.167:0/2699246509 shutdown_connections 2024-09-13T23:43:34.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.903+0000 7f91aceedf40 1 -- 172.21.15.167:0/2699246509 wait complete. 2024-09-13T23:43:34.918 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:34.920 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-09-13T23:43:34.921 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-09-13T23:43:34.921 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-09-13T23:43:34.964 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:34.964 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:34.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.955+0000 7fbff119bf40 1 Processor -- start 2024-09-13T23:43:34.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.955+0000 7fbff119bf40 1 -- start start 2024-09-13T23:43:34.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.955+0000 7fbff119bf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ca8a6ef000 0x55ca8a9d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.955+0000 7fbff119bf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ca8a7b3500 con 0x55ca8a6ef400 2024-09-13T23:43:34.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.955+0000 7fbff119bf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ca8a77ca00 con 0x55ca8a6ef000 2024-09-13T23:43:34.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.955+0000 7fbfef954700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ca8a6ef000 0x55ca8a9d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.955+0000 7fbfef153700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55ca8a6ef400 0x55ca8a7ee800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:45584/0 (socket says 172.21.15.167:45584) 2024-09-13T23:43:34.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.955+0000 7fbfef153700 1 -- 172.21.15.167:0/293481201 learned_addr learned my addr 172.21.15.167:0/293481201 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:34.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.955+0000 7fbfee952700 1 -- 172.21.15.167:0/293481201 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693671867 0 0) 0x55ca8a84a000 con 0x55ca8a6ef400 2024-09-13T23:43:34.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.955+0000 7fbfee952700 1 -- 172.21.15.167:0/293481201 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55ca8a7b3680 con 0x55ca8a6ef400 2024-09-13T23:43:34.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.955+0000 7fbfef954700 1 -- 172.21.15.167:0/293481201 >> v1:172.21.15.12:6789/0 conn(0x55ca8a6ef400 legacy=0x55ca8a7ee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.955+0000 7fbfef954700 1 -- 172.21.15.167:0/293481201 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca8a701d40 con 0x55ca8a6ef000 2024-09-13T23:43:34.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbfef954700 1 --2- 172.21.15.167:0/293481201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55ca8a9cfa70 tx=0x55ca8a7f3980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab99ca524cff2ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbfee952700 1 -- 172.21.15.167:0/293481201 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8a7934a0 con 0x55ca8a6ef000 2024-09-13T23:43:34.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbfee952700 1 -- 172.21.15.167:0/293481201 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca8a7b3680 con 0x55ca8a6ef000 2024-09-13T23:43:34.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbfee952700 1 -- 172.21.15.167:0/293481201 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8a793600 con 0x55ca8a6ef000 2024-09-13T23:43:34.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbff119bf40 1 -- 172.21.15.167:0/293481201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 msgr2=0x55ca8a9d4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbff119bf40 1 --2- 172.21.15.167:0/293481201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x55ca8a9cfa70 tx=0x55ca8a7f3980 comp rx=0 tx=0).stop 2024-09-13T23:43:34.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbff119bf40 1 -- 172.21.15.167:0/293481201 shutdown_connections 2024-09-13T23:43:34.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbff119bf40 1 --2- 172.21.15.167:0/293481201 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbff119bf40 1 -- 172.21.15.167:0/293481201 >> 172.21.15.167:0/293481201 conn(0x55ca8a6eec00 msgr2=0x55ca8a7ee000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbff119bf40 1 -- 172.21.15.167:0/293481201 shutdown_connections 2024-09-13T23:43:34.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbff119bf40 1 -- 172.21.15.167:0/293481201 wait complete. 2024-09-13T23:43:34.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbff119bf40 1 Processor -- start 2024-09-13T23:43:34.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbff119bf40 1 -- start start 2024-09-13T23:43:34.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbff119bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbff119bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8a77c500 con 0x55ca8a6ef000 2024-09-13T23:43:34.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbfef954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbfef954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53154/0 (socket says 172.21.15.167:53154) 2024-09-13T23:43:34.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbfef954700 1 -- 172.21.15.167:0/1421126724 learned_addr learned my addr 172.21.15.167:0/1421126724 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbfef954700 1 -- 172.21.15.167:0/1421126724 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca8a701ba0 con 0x55ca8a6ef000 2024-09-13T23:43:34.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.959+0000 7fbfef954700 1 --2- 172.21.15.167:0/1421126724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55ca8b4b9e00 tx=0x55ca8a7f3800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de810c6a5ac77eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbfee151700 1 -- 172.21.15.167:0/1421126724 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8a793760 con 0x55ca8a6ef000 2024-09-13T23:43:34.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbfee151700 1 -- 172.21.15.167:0/1421126724 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca8a7b3980 con 0x55ca8a6ef000 2024-09-13T23:43:34.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbfee151700 1 -- 172.21.15.167:0/1421126724 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8a7934a0 con 0x55ca8a6ef000 2024-09-13T23:43:34.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 -- 172.21.15.167:0/1421126724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 msgr2=0x55ca8a9d4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 --2- 172.21.15.167:0/1421126724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x55ca8b4b9e00 tx=0x55ca8a7f3800 comp rx=0 tx=0).stop 2024-09-13T23:43:34.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 -- 172.21.15.167:0/1421126724 shutdown_connections 2024-09-13T23:43:34.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 --2- 172.21.15.167:0/1421126724 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 -- 172.21.15.167:0/1421126724 >> 172.21.15.167:0/1421126724 conn(0x55ca8a6eec00 msgr2=0x55ca8a7ef000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 -- 172.21.15.167:0/1421126724 shutdown_connections 2024-09-13T23:43:34.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 -- 172.21.15.167:0/1421126724 wait complete. 2024-09-13T23:43:34.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 Processor -- start 2024-09-13T23:43:34.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 -- start start 2024-09-13T23:43:34.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8a77c3c0 con 0x55ca8a6ef000 2024-09-13T23:43:34.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbfef954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbfef954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53162/0 (socket says 172.21.15.167:53162) 2024-09-13T23:43:34.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbfef954700 1 -- 172.21.15.167:0/4034595229 learned_addr learned my addr 172.21.15.167:0/4034595229 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbfef954700 1 -- 172.21.15.167:0/4034595229 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca8a701a00 con 0x55ca8a6ef000 2024-09-13T23:43:34.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbfef954700 1 --2- 172.21.15.167:0/4034595229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x55ca8a9cff80 tx=0x55ca8a7f3980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbfec94e700 1 -- 172.21.15.167:0/4034595229 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8a793600 con 0x55ca8a6ef000 2024-09-13T23:43:34.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbfec94e700 1 -- 172.21.15.167:0/4034595229 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca8a7b3b00 con 0x55ca8a6ef000 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 -- 172.21.15.167:0/4034595229 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca8a701860 con 0x55ca8a6ef000 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbfec94e700 1 -- 172.21.15.167:0/4034595229 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8a7938c0 con 0x55ca8a6ef000 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.963+0000 7fbff119bf40 1 -- 172.21.15.167:0/4034595229 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca8a7016c0 con 0x55ca8a6ef000 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfec94e700 1 -- 172.21.15.167:0/4034595229 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ca8a7e8b00 con 0x55ca8a6ef000 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfec94e700 1 --2- 172.21.15.167:0/4034595229 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8aa8a000 0x55ca8a9d5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbff119bf40 1 Processor -- start 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfec94e700 1 -- 172.21.15.167:0/4034595229 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ca8a755180 con 0x55ca8a6ef000 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbff119bf40 1 -- start start 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfef153700 1 --2- 172.21.15.167:0/4034595229 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8aa8a000 0x55ca8a9d5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbff119bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 0x55ca8a9d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbff119bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8a714640 con 0x55ca8a6ef800 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfef954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 0x55ca8a9d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfef954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 0x55ca8a9d4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53176/0 (socket says 172.21.15.167:53176) 2024-09-13T23:43:34.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfef954700 1 -- 172.21.15.167:0/3486290075 learned_addr learned my addr 172.21.15.167:0/3486290075 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfef153700 1 --2- 172.21.15.167:0/4034595229 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8aa8a000 0x55ca8a9d5080 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55ca8b4b8030 tx=0x55ca8aa52b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfef954700 1 -- 172.21.15.167:0/3486290075 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca8a700000 con 0x55ca8a6ef800 2024-09-13T23:43:34.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfef954700 1 --2- 172.21.15.167:0/3486290075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 0x55ca8a9d4b00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55ca8b567ef0 tx=0x55ca8aa52c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849d038d2e368154 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfea149700 1 -- 172.21.15.167:0/3486290075 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8a793600 con 0x55ca8a6ef800 2024-09-13T23:43:34.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfea149700 1 -- 172.21.15.167:0/3486290075 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca8a7b3e00 con 0x55ca8a6ef800 2024-09-13T23:43:34.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.967+0000 7fbfea149700 1 -- 172.21.15.167:0/3486290075 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8a7938c0 con 0x55ca8a6ef800 2024-09-13T23:43:34.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbff119bf40 1 -- 172.21.15.167:0/3486290075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 msgr2=0x55ca8a9d4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:34.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbff119bf40 1 --2- 172.21.15.167:0/3486290075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 0x55ca8a9d4b00 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x55ca8b567ef0 tx=0x55ca8aa52c60 comp rx=0 tx=0).stop 2024-09-13T23:43:34.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbff119bf40 1 -- 172.21.15.167:0/3486290075 shutdown_connections 2024-09-13T23:43:34.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbff119bf40 1 --2- 172.21.15.167:0/3486290075 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 0x55ca8a9d4b00 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:34.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbff119bf40 1 -- 172.21.15.167:0/3486290075 >> 172.21.15.167:0/3486290075 conn(0x55ca8a6ef400 msgr2=0x55ca8a7ef800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:34.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbff119bf40 1 -- 172.21.15.167:0/3486290075 shutdown_connections 2024-09-13T23:43:34.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbff119bf40 1 -- 172.21.15.167:0/3486290075 wait complete. 2024-09-13T23:43:34.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbff119bf40 1 Processor -- start 2024-09-13T23:43:34.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbff119bf40 1 -- start start 2024-09-13T23:43:34.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbff119bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 0x55ca8a9d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbff119bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8a7148c0 con 0x55ca8a6ef800 2024-09-13T23:43:34.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbfef954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 0x55ca8a9d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbfef954700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 0x55ca8a9d4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53178/0 (socket says 172.21.15.167:53178) 2024-09-13T23:43:34.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 learned_addr learned my addr 172.21.15.167:0/207198463 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:34.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca8a700ea0 con 0x55ca8a6ef800 2024-09-13T23:43:34.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbfef954700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 0x55ca8a9d4b00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55ca8aaa8c60 tx=0x55ca8aa52e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbfe8946700 1 -- 172.21.15.167:0/207198463 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8a793760 con 0x55ca8a6ef800 2024-09-13T23:43:34.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbfe8946700 1 -- 172.21.15.167:0/207198463 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca8a7b3b00 con 0x55ca8a6ef800 2024-09-13T23:43:34.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbfe8946700 1 -- 172.21.15.167:0/207198463 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8a7934a0 con 0x55ca8a6ef800 2024-09-13T23:43:34.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.971+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca8a700820 con 0x55ca8a6ef800 2024-09-13T23:43:34.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca8a7009c0 con 0x55ca8a6ef800 2024-09-13T23:43:34.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbfe8946700 1 -- 172.21.15.167:0/207198463 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ca8a7e8dc0 con 0x55ca8a6ef800 2024-09-13T23:43:34.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbfe8946700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8aa8b000 0x55ca8a9d5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbfe8946700 1 -- 172.21.15.167:0/207198463 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ca8a755340 con 0x55ca8a6ef800 2024-09-13T23:43:34.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbfef153700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8aa8b000 0x55ca8a9d5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbff119bf40 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ca8aa8b400 0x55ca8a9d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8aa8b800 con 0x55ca8aa8b400 2024-09-13T23:43:34.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbfef153700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8aa8b000 0x55ca8a9d5600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ca8aaa9470 tx=0x55ca8aa52a80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbff0155700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ca8aa8b400 0x55ca8a9d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbff0155700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ca8aa8b400 0x55ca8a9d5b80 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:34.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbff119bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbff119bf40 20 realm 2024-09-13T23:43:34.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.975+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8aa8bc00 con 0x55ca8aa8b400 2024-09-13T23:43:34.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:34.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:34.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8aa8b800 con 0x55ca8aa8b400 2024-09-13T23:43:34.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:34.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:34.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ca8aa8bc00 0x55ca8a9d6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20000 con 0x55ca8aa8bc00 2024-09-13T23:43:34.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbfef954700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ca8aa8bc00 0x55ca8a9d6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbfef954700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ca8aa8bc00 0x55ca8a9d6100 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:34.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:34.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:34.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8bc00 2024-09-13T23:43:34.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:34.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:34.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 20 searching for the correct realm 2024-09-13T23:43:34.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ca8ab20800 0x55ca8a9d6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20c00 con 0x55ca8ab20800 2024-09-13T23:43:34.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.979+0000 7fbfef153700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ca8ab20800 0x55ca8a9d6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.983+0000 7fbfef153700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ca8ab20800 0x55ca8a9d6680 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.983+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:34.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.983+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20c00 con 0x55ca8aa8bc00 2024-09-13T23:43:34.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.983+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:34.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.983+0000 7fbfef954700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ca8ab20000 0x55ca8a9d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:34.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.983+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20000 2024-09-13T23:43:34.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.983+0000 7fbff0155700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ca8ab20000 0x55ca8a9d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:34.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.983+0000 7fbff0155700 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ca8ab20000 0x55ca8a9d6c00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:34.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.983+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:34.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.983+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8b400 2024-09-13T23:43:34.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.983+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:34.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.983+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8b400 2024-09-13T23:43:34.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:34.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20800 2024-09-13T23:43:34.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:34.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8b400 2024-09-13T23:43:34.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:34.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8bc00 2024-09-13T23:43:34.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:34.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8bc00 2024-09-13T23:43:34.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 33 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:34.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8b400 2024-09-13T23:43:34.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:34.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20000 2024-09-13T23:43:34.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:34.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.987+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8b400 2024-09-13T23:43:35.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20000 2024-09-13T23:43:35.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8bc00 2024-09-13T23:43:35.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20800 2024-09-13T23:43:35.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8b400 2024-09-13T23:43:35.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8bc00 2024-09-13T23:43:35.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20800 2024-09-13T23:43:35.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.991+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8b400 2024-09-13T23:43:35.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8b400 2024-09-13T23:43:35.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(25 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8b400 2024-09-13T23:43:35.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(26 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20800 2024-09-13T23:43:35.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20000 2024-09-13T23:43:35.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(28 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8b400 2024-09-13T23:43:35.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(29 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20000 2024-09-13T23:43:35.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(30 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.995+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20800 2024-09-13T23:43:35.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.999+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.999+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20000 2024-09-13T23:43:35.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.999+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(32 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.999+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8bc00 2024-09-13T23:43:35.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.999+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 33 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.999+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8bc00 2024-09-13T23:43:35.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.999+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.999+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20800 2024-09-13T23:43:35.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.999+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.999+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8bc00 2024-09-13T23:43:35.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:34.999+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 33 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 33 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8bc00 2024-09-13T23:43:35.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 33 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:35.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:35.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:35.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:35.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:35.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21000 con 0x55ca8aa8b400 2024-09-13T23:43:35.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:35.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:35.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21400 con 0x55ca8aa8bc00 2024-09-13T23:43:35.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:35.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:35.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21800 con 0x55ca8aa8bc00 2024-09-13T23:43:35.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:35.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 zone default found 2024-09-13T23:43:35.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 4 Realm: () 2024-09-13T23:43:35.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:35.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:35.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:35.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 zonegroup default 2024-09-13T23:43:35.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:35.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21c00 con 0x55ca8ab20000 2024-09-13T23:43:35.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:35.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:35.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.003+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21000 con 0x55ca8ab20800 2024-09-13T23:43:35.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.007+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.007+0000 7fbff119bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:35.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.007+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:35.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.007+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21c00 con 0x55ca8ab20800 2024-09-13T23:43:35.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.007+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.007+0000 7fbff119bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.007+0000 7fbff119bf40 20 started sync module instance, tier type = 2024-09-13T23:43:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.007+0000 7fbff119bf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:35.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.007+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8aa8b800 con 0x55ca8aa8bc00 2024-09-13T23:43:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.007+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v33'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.007+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94328397282048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20c00 con 0x55ca8aa8bc00 2024-09-13T23:43:35.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.007+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8aa8b400 2024-09-13T23:43:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.011+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(46 notify.1 [create] v33'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.011+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94328397283200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21400 con 0x55ca8aa8b400 2024-09-13T23:43:35.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.011+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21800 con 0x55ca8aa8bc00 2024-09-13T23:43:35.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.011+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94328397282048] v33'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.015+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94328397283200] v33'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.015+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v33'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.015+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94328397284352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21000 con 0x55ca8aa8bc00 2024-09-13T23:43:35.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.015+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21c00 con 0x55ca8ab20000 2024-09-13T23:43:35.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.019+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94328397284352] v33'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.019+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(50 notify.3 [create] v33'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.019+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94328397285504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8aa8a800 con 0x55ca8ab20000 2024-09-13T23:43:35.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.019+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8aa8b800 con 0x55ca8aa8bc00 2024-09-13T23:43:35.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.019+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(51 notify.3 [watch watch cookie 94328397285504] v33'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.019+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v33'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.019+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94328397286656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8aa8a800 con 0x55ca8aa8bc00 2024-09-13T23:43:35.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.019+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20800 2024-09-13T23:43:35.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.023+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v33'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.023+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94328397287808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21400 con 0x55ca8ab20800 2024-09-13T23:43:35.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.023+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21c00 con 0x55ca8ab20800 2024-09-13T23:43:35.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.023+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94328397286656] v33'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.023+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v33'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.023+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94328397288960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20400 con 0x55ca8ab20800 2024-09-13T23:43:35.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.023+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21400 con 0x55ca8ab20000 2024-09-13T23:43:35.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94328397287808] v33'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(58 notify.7 [create] v33'156 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94328397290112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21c00 con 0x55ca8ab20000 2024-09-13T23:43:35.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff119bf40 20 add_watcher() i=0 2024-09-13T23:43:35.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff119bf40 20 add_watcher() i=1 2024-09-13T23:43:35.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff119bf40 20 add_watcher() i=2 2024-09-13T23:43:35.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff119bf40 20 add_watcher() i=3 2024-09-13T23:43:35.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff119bf40 20 add_watcher() i=4 2024-09-13T23:43:35.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff119bf40 20 add_watcher() i=5 2024-09-13T23:43:35.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94328397288960] v33'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff119bf40 20 add_watcher() i=6 2024-09-13T23:43:35.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(59 notify.7 [watch watch cookie 94328397290112] v33'157 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff119bf40 20 add_watcher() i=7 2024-09-13T23:43:35.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff119bf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:35.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.027+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab34000 con 0x55ca8aa8b400 2024-09-13T23:43:35.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.031+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.031+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94328398774272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab34400 con 0x55ca8aa8b400 2024-09-13T23:43:35.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.031+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94328398774272] v33'150 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.031+0000 7fbfd6922700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:35.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.031+0000 7fbff119bf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:35.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.031+0000 7fbff119bf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:35.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.031+0000 7fbff119bf40 1 Processor -- start 2024-09-13T23:43:35.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.031+0000 7fbff119bf40 1 -- start start 2024-09-13T23:43:35.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.031+0000 7fbff119bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 0x55ca8a9d7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:35.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.031+0000 7fbff119bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8a77c3c0 con 0x55ca8ab34c00 2024-09-13T23:43:35.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbfef153700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 0x55ca8a9d7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:35.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbfef153700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 0x55ca8a9d7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53182/0 (socket says 172.21.15.167:53182) 2024-09-13T23:43:35.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbfef153700 1 -- 172.21.15.167:0/1273420806 learned_addr learned my addr 172.21.15.167:0/1273420806 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:35.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbfef153700 1 -- 172.21.15.167:0/1273420806 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca8a700b60 con 0x55ca8ab34c00 2024-09-13T23:43:35.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbfef153700 1 --2- 172.21.15.167:0/1273420806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 0x55ca8a9d7180 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55ca8aac5d10 tx=0x55ca8aa53ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e64032176836d02 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:35.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbfd6121700 1 -- 172.21.15.167:0/1273420806 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8ab36160 con 0x55ca8ab34c00 2024-09-13T23:43:35.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbfd6121700 1 -- 172.21.15.167:0/1273420806 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca8ab44180 con 0x55ca8ab34c00 2024-09-13T23:43:35.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbfd6121700 1 -- 172.21.15.167:0/1273420806 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8ab362c0 con 0x55ca8ab34c00 2024-09-13T23:43:35.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbff119bf40 1 -- 172.21.15.167:0/1273420806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 msgr2=0x55ca8a9d7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:35.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbff119bf40 1 --2- 172.21.15.167:0/1273420806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 0x55ca8a9d7180 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x55ca8aac5d10 tx=0x55ca8aa53ec0 comp rx=0 tx=0).stop 2024-09-13T23:43:35.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbff119bf40 1 -- 172.21.15.167:0/1273420806 shutdown_connections 2024-09-13T23:43:35.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbff119bf40 1 --2- 172.21.15.167:0/1273420806 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 0x55ca8a9d7180 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:35.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbff119bf40 1 -- 172.21.15.167:0/1273420806 >> 172.21.15.167:0/1273420806 conn(0x55ca8ab34800 msgr2=0x55ca8a7f0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:35.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbff119bf40 1 -- 172.21.15.167:0/1273420806 shutdown_connections 2024-09-13T23:43:35.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbff119bf40 1 -- 172.21.15.167:0/1273420806 wait complete. 2024-09-13T23:43:35.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbff119bf40 1 Processor -- start 2024-09-13T23:43:35.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbff119bf40 1 -- start start 2024-09-13T23:43:35.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbff119bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 0x55ca8a9d7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:35.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbff119bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8a77c500 con 0x55ca8ab34c00 2024-09-13T23:43:35.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbfef153700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 0x55ca8a9d7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:35.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbfef153700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 0x55ca8a9d7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:53192/0 (socket says 172.21.15.167:53192) 2024-09-13T23:43:35.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.035+0000 7fbfef153700 1 -- 172.21.15.167:0/1740027432 learned_addr learned my addr 172.21.15.167:0/1740027432 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:35.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbfef153700 1 -- 172.21.15.167:0/1740027432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca8a700d00 con 0x55ca8ab34c00 2024-09-13T23:43:35.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbfef153700 1 --2- 172.21.15.167:0/1740027432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 0x55ca8a9d7180 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x55ca8ab4a150 tx=0x55ca8aa52c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:35.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbfd491e700 1 -- 172.21.15.167:0/1740027432 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8ab36420 con 0x55ca8ab34c00 2024-09-13T23:43:35.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbfd491e700 1 -- 172.21.15.167:0/1740027432 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca8ab44300 con 0x55ca8ab34c00 2024-09-13T23:43:35.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbfd491e700 1 -- 172.21.15.167:0/1740027432 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ca8ab36160 con 0x55ca8ab34c00 2024-09-13T23:43:35.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbff119bf40 1 -- 172.21.15.167:0/1740027432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca8a700340 con 0x55ca8ab34c00 2024-09-13T23:43:35.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbff119bf40 1 -- 172.21.15.167:0/1740027432 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca8a700b60 con 0x55ca8ab34c00 2024-09-13T23:43:35.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbfd491e700 1 -- 172.21.15.167:0/1740027432 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ca8a7e98c0 con 0x55ca8ab34c00 2024-09-13T23:43:35.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbfd491e700 1 --2- 172.21.15.167:0/1740027432 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8abc4000 0x55ca8abca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:35.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbfd491e700 1 -- 172.21.15.167:0/1740027432 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6061+0+0 (secure 0 0 0) 0x55ca8a7556c0 con 0x55ca8ab34c00 2024-09-13T23:43:35.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbff119bf40 5 note: GC not initialized 2024-09-13T23:43:35.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbff0155700 1 --2- 172.21.15.167:0/1740027432 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8abc4000 0x55ca8abca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:35.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.039+0000 7fbff0155700 1 --2- 172.21.15.167:0/1740027432 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8abc4000 0x55ca8abca000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55ca8aac5140 tx=0x55ca8ab12120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:35.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:35.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:35.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbfd0115700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:05 2024 2024-09-13T23:43:35.052 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbfcf8f3700 20 reqs_thread_entry: start 2024-09-13T23:43:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab35800 con 0x55ca8aa8b400 2024-09-13T23:43:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbfd2119700 20 reqs_thread_entry: start 2024-09-13T23:43:35.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv12 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:35.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:35.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:35.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:35.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab35c00 con 0x55ca8ab20000 2024-09-13T23:43:35.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 20 enabling bucket name=myfoo 2024-09-13T23:43:35.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-13T23:43:35.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:35.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.043+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8abc4400 con 0x55ca8ab20000 2024-09-13T23:43:35.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+165 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:35.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-09-13T23:43:35.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 : miss 2024-09-13T23:43:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8abc4800 con 0x55ca8ab20800 2024-09-13T23:43:35.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(65 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv6 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 2024-09-13T23:43:35.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.047+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:66 8.0 8:008769f0:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8abc5000 con 0x55ca8ab20800 2024-09-13T23:43:35.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(66 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 [delete,create,call,call,writefull 0~352,setxattr (123)] v33'7 uv7 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbff119bf40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:35.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbff119bf40 10 moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:35.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbff119bf40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:35.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbff119bf40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1, ofs0, ns] 2024-09-13T23:43:35.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94328398776576 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8abc5800 con 0x55ca8aa8bc00 2024-09-13T23:43:35.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94328397282048 notify 141733920799 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x55ca8a7016c0 con 0x55ca8aa8bc00 2024-09-13T23:43:35.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbfe9948700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920799 cookie 94328397282048 notifier 4646 bl.length()=725 2024-09-13T23:43:35.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.0 [notify cookie 94328398776576 out=8b] v0'0 uv156 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbfe9948700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 info.flags=0x17 2024-09-13T23:43:35.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbfe9948700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1 to cache LRU end 2024-09-13T23:43:35.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbfe9948700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:35.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbfe9948700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8abc4800 con 0x55ca8aa8bc00 2024-09-13T23:43:35.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== watch-notify(notify_complete (2) cookie 94328398776576 notify 141733920799 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ca8a701d40 con 0x55ca8aa8bc00 2024-09-13T23:43:35.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbff119bf40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): orig_sources= new_sources= 2024-09-13T23:43:35.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbff119bf40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): potential sources added=[] removed=[] 2024-09-13T23:43:35.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbff119bf40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): orig_dests= new_dests= 2024-09-13T23:43:35.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbff119bf40 20 handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1]): potential dests added=[] removed=[] 2024-09-13T23:43:35.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.051+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab34000 con 0x55ca8aa8b400 2024-09-13T23:43:35.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~357] v33'13 uv13 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbff119bf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:35.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbff119bf40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:35.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbff119bf40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:35.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94328398776576 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab34400 con 0x55ca8aa8bc00 2024-09-13T23:43:35.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== watch-notify(notify (1) cookie 94328397286656 notify 141733920800 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x55ca8a701d40 con 0x55ca8aa8bc00 2024-09-13T23:43:35.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(70 notify.4 [notify cookie 94328398776576 out=8b] v0'0 uv156 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbfe9948700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 141733920800 cookie 94328397286656 notifier 4646 bl.length()=533 2024-09-13T23:43:35.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbfe9948700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:35.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbfe9948700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:35.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbfe9948700 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8abc5400 con 0x55ca8aa8bc00 2024-09-13T23:43:35.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== watch-notify(notify_complete (2) cookie 94328398776576 notify 141733920800 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55ca8a701ba0 con 0x55ca8aa8bc00 2024-09-13T23:43:35.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.055+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.067 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:35.067 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:35.067 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:35.067 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:35.067 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:35.067 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:35.067 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:35.067 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:35.067 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:35.067 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:35.067 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:35.068 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:35.069 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:35.070 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:35.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.059+0000 7fbfd6922700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:35.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.059+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94328398774272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21400 con 0x55ca8aa8b400 2024-09-13T23:43:35.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.059+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94328398774272] v33'151 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.063+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94328397282048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab21c00 con 0x55ca8aa8bc00 2024-09-13T23:43:35.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.063+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94328397282048] v33'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.063+0000 7fbff119bf40 20 remove_watcher() i=0 2024-09-13T23:43:35.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.063+0000 7fbff119bf40 2 removed watcher, disabling cache 2024-09-13T23:43:35.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.063+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94328397283200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8abc5800 con 0x55ca8aa8b400 2024-09-13T23:43:35.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.067+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94328397283200] v33'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8aa8b400 2024-09-13T23:43:35.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.067+0000 7fbff119bf40 20 remove_watcher() i=1 2024-09-13T23:43:35.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.067+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94328397284352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8abc4800 con 0x55ca8aa8bc00 2024-09-13T23:43:35.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.071+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94328397284352] v33'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.071+0000 7fbff119bf40 20 remove_watcher() i=2 2024-09-13T23:43:35.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.071+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94328397285504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8abc4c00 con 0x55ca8ab20000 2024-09-13T23:43:35.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.071+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94328397285504] v33'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.075+0000 7fbff119bf40 20 remove_watcher() i=3 2024-09-13T23:43:35.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.075+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94328397286656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8abc5000 con 0x55ca8aa8bc00 2024-09-13T23:43:35.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.075+0000 7fbfef954700 1 -- 172.21.15.167:0/207198463 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94328397286656] v33'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d200 con 0x55ca8aa8bc00 2024-09-13T23:43:35.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.075+0000 7fbff119bf40 20 remove_watcher() i=4 2024-09-13T23:43:35.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.075+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94328397287808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8aa8b800 con 0x55ca8ab20800 2024-09-13T23:43:35.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.079+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94328397287808] v33'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.079+0000 7fbff119bf40 20 remove_watcher() i=5 2024-09-13T23:43:35.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.079+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94328397288960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8aa8a800 con 0x55ca8ab20800 2024-09-13T23:43:35.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.079+0000 7fbfef153700 1 -- 172.21.15.167:0/207198463 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94328397288960] v33'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1d440 con 0x55ca8ab20800 2024-09-13T23:43:35.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.079+0000 7fbff119bf40 20 remove_watcher() i=6 2024-09-13T23:43:35.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.079+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94328397290112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33) v8 -- 0x55ca8ab20c00 con 0x55ca8ab20000 2024-09-13T23:43:35.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94328397290112] v33'158 uv156 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ca8ab1cb40 con 0x55ca8ab20000 2024-09-13T23:43:35.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 20 remove_watcher() i=7 2024-09-13T23:43:35.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ca8aa8bc00 msgr2=0x55ca8a9d6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:35.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ca8aa8bc00 0x55ca8a9d6100 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:35.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ca8aa8b400 msgr2=0x55ca8a9d5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:35.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ca8aa8b400 0x55ca8a9d5b80 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:35.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ca8ab20000 msgr2=0x55ca8a9d6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:35.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ca8ab20000 0x55ca8a9d6c00 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:35.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ca8ab20800 msgr2=0x55ca8a9d6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:35.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ca8ab20800 0x55ca8a9d6680 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:35.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8aa8b000 msgr2=0x55ca8a9d5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:35.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8aa8b000 0x55ca8a9d5600 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ca8aaa9470 tx=0x55ca8aa52a80 comp rx=0 tx=0).stop 2024-09-13T23:43:35.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 msgr2=0x55ca8a9d4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:35.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/207198463 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef800 0x55ca8a9d4b00 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x55ca8aaa8c60 tx=0x55ca8aa52e40 comp rx=0 tx=0).stop 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff0155700 1 -- 172.21.15.167:0/207198463 reap_dead start 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 shutdown_connections 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 >> 172.21.15.167:0/207198463 conn(0x55ca8a6ef400 msgr2=0x55ca8a7ef800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 shutdown_connections 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/207198463 wait complete. 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/1740027432 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8abc4000 msgr2=0x55ca8abca000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/1740027432 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8abc4000 0x55ca8abca000 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x55ca8aac5140 tx=0x55ca8ab12120 comp rx=0 tx=0).stop 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/1740027432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 msgr2=0x55ca8a9d7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/1740027432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 0x55ca8a9d7180 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x55ca8ab4a150 tx=0x55ca8aa52c00 comp rx=0 tx=0).stop 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/1740027432 shutdown_connections 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/1740027432 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8abc4000 0x55ca8abca000 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/1740027432 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8ab34c00 0x55ca8a9d7180 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/1740027432 >> 172.21.15.167:0/1740027432 conn(0x55ca8ab34800 msgr2=0x55ca8a7f0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:35.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/1740027432 shutdown_connections 2024-09-13T23:43:35.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/1740027432 wait complete. 2024-09-13T23:43:35.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/4034595229 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8aa8a000 msgr2=0x55ca8a9d5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:35.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/4034595229 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8aa8a000 0x55ca8a9d5080 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55ca8b4b8030 tx=0x55ca8aa52b40 comp rx=0 tx=0).stop 2024-09-13T23:43:35.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/4034595229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 msgr2=0x55ca8a9d4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:35.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/4034595229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x55ca8a9cff80 tx=0x55ca8a7f3980 comp rx=0 tx=0).stop 2024-09-13T23:43:35.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/4034595229 shutdown_connections 2024-09-13T23:43:35.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/4034595229 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ca8aa8a000 0x55ca8a9d5080 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:35.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 --2- 172.21.15.167:0/4034595229 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ca8a6ef000 0x55ca8a9d4580 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:35.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/4034595229 >> 172.21.15.167:0/4034595229 conn(0x55ca8a6eec00 msgr2=0x55ca8a7ef000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:35.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/4034595229 shutdown_connections 2024-09-13T23:43:35.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:35.083+0000 7fbff119bf40 1 -- 172.21.15.167:0/4034595229 wait complete. 2024-09-13T23:43:35.097 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:35.117 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-09-13T23:43:35.387 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=30000000 bytes_in=0 failed=False 2024-09-13T23:43:35.401 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:50.402 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-09-13T23:43:50.402 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-09-13T23:43:50.403 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-09-13T23:43:50.445 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:50.445 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:50.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.435+0000 7fd8ca3bcf40 1 Processor -- start 2024-09-13T23:43:50.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.435+0000 7fd8ca3bcf40 1 -- start start 2024-09-13T23:43:50.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.435+0000 7fd8ca3bcf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x560f9b9a9000 0x560f9bc8e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.435+0000 7fd8ca3bcf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x560f9ba6d500 con 0x560f9b9a9400 2024-09-13T23:43:50.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.435+0000 7fd8ca3bcf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x560f9ba36a00 con 0x560f9b9a9000 2024-09-13T23:43:50.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.435+0000 7fd8c8b75700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x560f9b9a9000 0x560f9bc8e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.435+0000 7fd8c8374700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x560f9b9a9400 0x560f9baa8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:49302/0 (socket says 172.21.15.167:49302) 2024-09-13T23:43:50.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.435+0000 7fd8c8374700 1 -- 172.21.15.167:0/2769055535 learned_addr learned my addr 172.21.15.167:0/2769055535 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:50.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2769055535 >> v1:172.21.15.12:6789/0 conn(0x560f9b9a9400 legacy=0x560f9baa8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2769055535 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f9b9bbd40 con 0x560f9b9a9000 2024-09-13T23:43:50.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8c8b75700 1 --2- 172.21.15.167:0/2769055535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9000 0x560f9bc8e580 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x560f9bc89a70 tx=0x560f9baad920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171e97545648554b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8c7b73700 1 -- 172.21.15.167:0/2769055535 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9ba4d4a0 con 0x560f9b9a9000 2024-09-13T23:43:50.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8c7b73700 1 -- 172.21.15.167:0/2769055535 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f9ba6d800 con 0x560f9b9a9000 2024-09-13T23:43:50.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8c7b73700 1 -- 172.21.15.167:0/2769055535 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9ba4d600 con 0x560f9b9a9000 2024-09-13T23:43:50.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2769055535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9000 msgr2=0x560f9bc8e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2769055535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9000 0x560f9bc8e580 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x560f9bc89a70 tx=0x560f9baad920 comp rx=0 tx=0).stop 2024-09-13T23:43:50.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2769055535 shutdown_connections 2024-09-13T23:43:50.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2769055535 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9000 0x560f9bc8e580 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2769055535 >> 172.21.15.167:0/2769055535 conn(0x560f9b9a8c00 msgr2=0x560f9baa8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:50.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2769055535 shutdown_connections 2024-09-13T23:43:50.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2769055535 wait complete. 2024-09-13T23:43:50.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8ca3bcf40 1 Processor -- start 2024-09-13T23:43:50.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8ca3bcf40 1 -- start start 2024-09-13T23:43:50.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8ca3bcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 0x560f9bc8eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8ca3bcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9ba36500 con 0x560f9b9a9400 2024-09-13T23:43:50.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8c8b75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 0x560f9bc8eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8c8b75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 0x560f9bc8eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52070/0 (socket says 172.21.15.167:52070) 2024-09-13T23:43:50.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8c8b75700 1 -- 172.21.15.167:0/3784696516 learned_addr learned my addr 172.21.15.167:0/3784696516 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:50.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.439+0000 7fd8c8b75700 1 -- 172.21.15.167:0/3784696516 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f9b9bbba0 con 0x560f9b9a9400 2024-09-13T23:43:50.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8c8b75700 1 --2- 172.21.15.167:0/3784696516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 0x560f9bc8eb00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x560f9c80d8c0 tx=0x560f9baad800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0670f85dabf22ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8c7372700 1 -- 172.21.15.167:0/3784696516 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9ba4d760 con 0x560f9b9a9400 2024-09-13T23:43:50.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8c7372700 1 -- 172.21.15.167:0/3784696516 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f9ba6db00 con 0x560f9b9a9400 2024-09-13T23:43:50.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8c7372700 1 -- 172.21.15.167:0/3784696516 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9ba4d4a0 con 0x560f9b9a9400 2024-09-13T23:43:50.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/3784696516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 msgr2=0x560f9bc8eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/3784696516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 0x560f9bc8eb00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x560f9c80d8c0 tx=0x560f9baad800 comp rx=0 tx=0).stop 2024-09-13T23:43:50.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/3784696516 shutdown_connections 2024-09-13T23:43:50.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/3784696516 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 0x560f9bc8eb00 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/3784696516 >> 172.21.15.167:0/3784696516 conn(0x560f9b9a8c00 msgr2=0x560f9baa9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:50.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/3784696516 shutdown_connections 2024-09-13T23:43:50.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/3784696516 wait complete. 2024-09-13T23:43:50.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8ca3bcf40 1 Processor -- start 2024-09-13T23:43:50.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8ca3bcf40 1 -- start start 2024-09-13T23:43:50.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8ca3bcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 0x560f9bc8eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.443+0000 7fd8ca3bcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9ba363c0 con 0x560f9b9a9400 2024-09-13T23:43:50.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c8b75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 0x560f9bc8eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c8b75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 0x560f9bc8eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52074/0 (socket says 172.21.15.167:52074) 2024-09-13T23:43:50.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2776512916 learned_addr learned my addr 172.21.15.167:0/2776512916 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:50.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2776512916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f9b9bba00 con 0x560f9b9a9400 2024-09-13T23:43:50.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c8b75700 1 --2- 172.21.15.167:0/2776512916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 0x560f9bc8eb00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x560f9c80df80 tx=0x560f9baad920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c5b6f700 1 -- 172.21.15.167:0/2776512916 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9ba4d600 con 0x560f9b9a9400 2024-09-13T23:43:50.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c5b6f700 1 -- 172.21.15.167:0/2776512916 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f9ba6dc80 con 0x560f9b9a9400 2024-09-13T23:43:50.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2776512916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f9b9bb860 con 0x560f9b9a9400 2024-09-13T23:43:50.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c5b6f700 1 -- 172.21.15.167:0/2776512916 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9ba4d8c0 con 0x560f9b9a9400 2024-09-13T23:43:50.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2776512916 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f9b9bb6c0 con 0x560f9b9a9400 2024-09-13T23:43:50.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c5b6f700 1 -- 172.21.15.167:0/2776512916 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x560f9baa2b00 con 0x560f9b9a9400 2024-09-13T23:43:50.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8ca3bcf40 1 Processor -- start 2024-09-13T23:43:50.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8ca3bcf40 1 -- start start 2024-09-13T23:43:50.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c5b6f700 1 --2- 172.21.15.167:0/2776512916 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9b9a9c00 0x560f9bc8f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c5b6f700 1 -- 172.21.15.167:0/2776512916 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x560f9ba0f180 con 0x560f9b9a9400 2024-09-13T23:43:50.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8ca3bcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 0x560f9bc8f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8ca3bcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9ba36000 con 0x560f9bd44000 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c9376700 1 --2- 172.21.15.167:0/2776512916 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9b9a9c00 0x560f9bc8f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c8b75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 0x560f9bc8f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c8b75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 0x560f9bc8f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52088/0 (socket says 172.21.15.167:52088) 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.447+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2445492952 learned_addr learned my addr 172.21.15.167:0/2445492952 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2445492952 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f9b9ba000 con 0x560f9bd44000 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8c9376700 1 --2- 172.21.15.167:0/2776512916 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9b9a9c00 0x560f9bc8f080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x560f9c81c8d0 tx=0x560f9bd0cb40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8c8b75700 1 --2- 172.21.15.167:0/2445492952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 0x560f9bc8f600 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x560f9c81d0e0 tx=0x560f9bd0cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109be14491557dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8c336a700 1 -- 172.21.15.167:0/2445492952 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9ba4d600 con 0x560f9bd44000 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8c336a700 1 -- 172.21.15.167:0/2445492952 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f9ba6dc80 con 0x560f9bd44000 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8c336a700 1 -- 172.21.15.167:0/2445492952 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9ba4d8c0 con 0x560f9bd44000 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2445492952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 msgr2=0x560f9bc8f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2445492952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 0x560f9bc8f600 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x560f9c81d0e0 tx=0x560f9bd0cc60 comp rx=0 tx=0).stop 2024-09-13T23:43:50.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2445492952 shutdown_connections 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2445492952 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 0x560f9bc8f600 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2445492952 >> 172.21.15.167:0/2445492952 conn(0x560f9b9a9800 msgr2=0x560f9baa9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2445492952 shutdown_connections 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2445492952 wait complete. 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8ca3bcf40 1 Processor -- start 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8ca3bcf40 1 -- start start 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8ca3bcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 0x560f9bc8f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8ca3bcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9b9ce640 con 0x560f9bd44000 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8c8b75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 0x560f9bc8f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8c8b75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 0x560f9bc8f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52102/0 (socket says 172.21.15.167:52102) 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.451+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 learned_addr learned my addr 172.21.15.167:0/1358080846 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f9b9baea0 con 0x560f9bd44000 2024-09-13T23:43:50.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8c8b75700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 0x560f9bc8f600 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x560f9c81c6c0 tx=0x560f9bd0ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8c1b67700 1 -- 172.21.15.167:0/1358080846 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9ba4d760 con 0x560f9bd44000 2024-09-13T23:43:50.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8c1b67700 1 -- 172.21.15.167:0/1358080846 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f9bd72000 con 0x560f9bd44000 2024-09-13T23:43:50.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f9b9ba820 con 0x560f9bd44000 2024-09-13T23:43:50.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8c1b67700 1 -- 172.21.15.167:0/1358080846 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9ba4d4a0 con 0x560f9bd44000 2024-09-13T23:43:50.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f9b9ba9c0 con 0x560f9bd44000 2024-09-13T23:43:50.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8ca3bcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8c1b67700 1 -- 172.21.15.167:0/1358080846 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x560f9baa2dc0 con 0x560f9bd44000 2024-09-13T23:43:50.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8c1b67700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9bd45800 0x560f9bc90680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8c1b67700 1 -- 172.21.15.167:0/1358080846 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x560f9ba0f340 con 0x560f9bd44000 2024-09-13T23:43:50.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8c8374700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9bd45800 0x560f9bc90680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560f9bd44400 0x560f9bc8fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.455+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd44800 con 0x560f9bd44400 2024-09-13T23:43:50.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8c9376700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560f9bd44400 0x560f9bc8fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8c8374700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9bd45800 0x560f9bc90680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x560f9bd67d10 tx=0x560f9bddc000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8c9376700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560f9bd44400 0x560f9bc8fb80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8ca3bcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8ca3bcf40 20 realm 2024-09-13T23:43:50.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8ca3bcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd44c00 con 0x560f9bd44400 2024-09-13T23:43:50.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8ca3bcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8ca3bcf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:50.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8ca3bcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd44800 con 0x560f9bd44400 2024-09-13T23:43:50.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8ca3bcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8ca3bcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560f9bd44c00 0x560f9bc90100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd45c00 con 0x560f9bd44c00 2024-09-13T23:43:50.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8c8b75700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560f9bd44c00 0x560f9bc90100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.459+0000 7fd8c8b75700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560f9bd44c00 0x560f9bc90100 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8ca3bcf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:50.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8ca3bcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce000 con 0x560f9bd44c00 2024-09-13T23:43:50.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8ca3bcf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:50.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8ca3bcf40 20 searching for the correct realm 2024-09-13T23:43:50.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560f9bdce400 0x560f9bc90c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce400 2024-09-13T23:43:50.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8c8374700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560f9bdce400 0x560f9bc90c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8c8374700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560f9bdce400 0x560f9bc90c00 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44c00 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8c8b75700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560f9bdce000 0x560f9bc8e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.463+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce000 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.467+0000 7fd8c9376700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560f9bdce000 0x560f9bc8e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.467+0000 7fd8c9376700 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560f9bdce000 0x560f9bc8e580 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.467+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.467+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44400 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.467+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.467+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44400 2024-09-13T23:43:50.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.467+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.467+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce400 2024-09-13T23:43:50.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.467+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.467+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44400 2024-09-13T23:43:50.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.467+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.467+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44c00 2024-09-13T23:43:50.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44c00 2024-09-13T23:43:50.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44400 2024-09-13T23:43:50.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce000 2024-09-13T23:43:50.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44400 2024-09-13T23:43:50.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce000 2024-09-13T23:43:50.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44c00 2024-09-13T23:43:50.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.471+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce400 2024-09-13T23:43:50.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.475+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.475+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44400 2024-09-13T23:43:50.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.475+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.475+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44c00 2024-09-13T23:43:50.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.475+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.475+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce400 2024-09-13T23:43:50.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.475+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.475+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44400 2024-09-13T23:43:50.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.475+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.475+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce000 2024-09-13T23:43:50.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.475+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.475+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44400 2024-09-13T23:43:50.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce400 2024-09-13T23:43:50.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce000 2024-09-13T23:43:50.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44400 2024-09-13T23:43:50.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce000 2024-09-13T23:43:50.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce400 2024-09-13T23:43:50.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce000 2024-09-13T23:43:50.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.479+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44c00 2024-09-13T23:43:50.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44c00 2024-09-13T23:43:50.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdce400 2024-09-13T23:43:50.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.492 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44c00 2024-09-13T23:43:50.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.493 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44c00 2024-09-13T23:43:50.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8ca3bcf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:50.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8ca3bcf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:50.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8ca3bcf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:50.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8ca3bcf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:50.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8ca3bcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.483+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9bd44400 2024-09-13T23:43:50.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.495 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf000 con 0x560f9bd44c00 2024-09-13T23:43:50.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:50.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf400 con 0x560f9bd44c00 2024-09-13T23:43:50.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:50.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 zone default found 2024-09-13T23:43:50.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 4 Realm: () 2024-09-13T23:43:50.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:50.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:50.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:50.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 zonegroup default 2024-09-13T23:43:50.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf800 con 0x560f9bdce000 2024-09-13T23:43:50.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9bdce400 2024-09-13T23:43:50.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9bdce400 2024-09-13T23:43:50.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 started sync module instance, tier type = 2024-09-13T23:43:50.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:50.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.487+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf800 con 0x560f9bd44c00 2024-09-13T23:43:50.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.491+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.491+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94625038097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd44800 con 0x560f9bd44c00 2024-09-13T23:43:50.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.491+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44400 2024-09-13T23:43:50.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.495+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.495+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94625038098304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf000 con 0x560f9bd44400 2024-09-13T23:43:50.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.495+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94625038097152] v35'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.495+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf400 con 0x560f9bd44c00 2024-09-13T23:43:50.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.499+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94625038098304] v35'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.499+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.499+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94625038099456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf800 con 0x560f9bd44c00 2024-09-13T23:43:50.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.499+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9bdce000 2024-09-13T23:43:50.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.499+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.503+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94625038100608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9bdce000 2024-09-13T23:43:50.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.503+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd45c00 con 0x560f9bd44c00 2024-09-13T23:43:50.511 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.503+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94625038099456] v35'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.503+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94625038100608] v35'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.503+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.503+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94625038101760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bd44c00 2024-09-13T23:43:50.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.503+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf000 con 0x560f9bdce400 2024-09-13T23:43:50.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.507+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.507+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94625038102912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9bdce400 2024-09-13T23:43:50.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.507+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9bdce400 2024-09-13T23:43:50.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.507+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94625038101760] v35'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.507+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94625038102912] v35'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.507+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.507+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94625038104064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf000 con 0x560f9bdce400 2024-09-13T23:43:50.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.507+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9bdce000 2024-09-13T23:43:50.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.511+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94625038104064] v35'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'159 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94625038105216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9bdce000 2024-09-13T23:43:50.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8ca3bcf40 20 add_watcher() i=0 2024-09-13T23:43:50.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8ca3bcf40 20 add_watcher() i=1 2024-09-13T23:43:50.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8ca3bcf40 20 add_watcher() i=2 2024-09-13T23:43:50.523 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8ca3bcf40 20 add_watcher() i=3 2024-09-13T23:43:50.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8ca3bcf40 20 add_watcher() i=4 2024-09-13T23:43:50.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8ca3bcf40 20 add_watcher() i=5 2024-09-13T23:43:50.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8ca3bcf40 20 add_watcher() i=6 2024-09-13T23:43:50.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94625038105216] v35'160 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8ca3bcf40 20 add_watcher() i=7 2024-09-13T23:43:50.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8ca3bcf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:50.525 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4000 con 0x560f9bd44400 2024-09-13T23:43:50.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.526 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.515+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94625039613952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4400 con 0x560f9bd44400 2024-09-13T23:43:50.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94625039613952] v35'152 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8afb43700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:50.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8ca3bcf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:50.528 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8ca3bcf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:50.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8ca3bcf40 1 Processor -- start 2024-09-13T23:43:50.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8ca3bcf40 1 -- start start 2024-09-13T23:43:50.529 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8ca3bcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 0x560f9bc91180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8ca3bcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9b9ce780 con 0x560f9bdf4c00 2024-09-13T23:43:50.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8c8374700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 0x560f9bc91180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8c8374700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 0x560f9bc91180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52116/0 (socket says 172.21.15.167:52116) 2024-09-13T23:43:50.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8c8374700 1 -- 172.21.15.167:0/3024269880 learned_addr learned my addr 172.21.15.167:0/3024269880 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:50.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8c8374700 1 -- 172.21.15.167:0/3024269880 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f9b9bab60 con 0x560f9bdf4c00 2024-09-13T23:43:50.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.519+0000 7fd8c8374700 1 --2- 172.21.15.167:0/3024269880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 0x560f9bc91180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x560f9be00090 tx=0x560f9bd0df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b0f6b589689507 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8af342700 1 -- 172.21.15.167:0/3024269880 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9bdf6160 con 0x560f9bdf4c00 2024-09-13T23:43:50.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8af342700 1 -- 172.21.15.167:0/3024269880 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f9bd72a80 con 0x560f9bdf4c00 2024-09-13T23:43:50.531 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8af342700 1 -- 172.21.15.167:0/3024269880 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9bdf62c0 con 0x560f9bdf4c00 2024-09-13T23:43:50.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/3024269880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 msgr2=0x560f9bc91180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/3024269880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 0x560f9bc91180 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x560f9be00090 tx=0x560f9bd0df20 comp rx=0 tx=0).stop 2024-09-13T23:43:50.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/3024269880 shutdown_connections 2024-09-13T23:43:50.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/3024269880 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 0x560f9bc91180 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/3024269880 >> 172.21.15.167:0/3024269880 conn(0x560f9bdf4800 msgr2=0x560f9baaa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:50.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/3024269880 shutdown_connections 2024-09-13T23:43:50.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/3024269880 wait complete. 2024-09-13T23:43:50.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8ca3bcf40 1 Processor -- start 2024-09-13T23:43:50.533 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8ca3bcf40 1 -- start start 2024-09-13T23:43:50.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8ca3bcf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 0x560f9bc91180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8ca3bcf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f9ba36000 con 0x560f9bdf4c00 2024-09-13T23:43:50.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8c8374700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 0x560f9bc91180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8c8374700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 0x560f9bc91180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52122/0 (socket says 172.21.15.167:52122) 2024-09-13T23:43:50.534 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8c8374700 1 -- 172.21.15.167:0/2227567392 learned_addr learned my addr 172.21.15.167:0/2227567392 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:50.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8c8374700 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f9b9bad00 con 0x560f9bdf4c00 2024-09-13T23:43:50.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.523+0000 7fd8c8374700 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 0x560f9bc91180 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x560f9be00d50 tx=0x560f9bddc0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8adb3f700 1 -- 172.21.15.167:0/2227567392 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9bdf6420 con 0x560f9bdf4c00 2024-09-13T23:43:50.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8adb3f700 1 -- 172.21.15.167:0/2227567392 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f9bd72780 con 0x560f9bdf4c00 2024-09-13T23:43:50.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f9b9ba340 con 0x560f9bdf4c00 2024-09-13T23:43:50.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8adb3f700 1 -- 172.21.15.167:0/2227567392 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x560f9bdf6160 con 0x560f9bdf4c00 2024-09-13T23:43:50.536 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f9b9bab60 con 0x560f9bdf4c00 2024-09-13T23:43:50.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8adb3f700 1 -- 172.21.15.167:0/2227567392 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x560f9baa3b80 con 0x560f9bdf4c00 2024-09-13T23:43:50.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8adb3f700 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9be84000 0x560f9be8a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8adb3f700 1 -- 172.21.15.167:0/2227567392 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x560f9ba0f6c0 con 0x560f9bdf4c00 2024-09-13T23:43:50.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8c9376700 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9be84000 0x560f9be8a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560f9bdf5000 0x560f9bc91700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.537 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf5400 con 0x560f9bdf5000 2024-09-13T23:43:50.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8c8b75700 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560f9bdf5000 0x560f9bc91700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8c9376700 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9be84000 0x560f9be8a000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x560f9bd8fb00 tx=0x560f9bd0ca80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.527+0000 7fd8c8b75700 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560f9bdf5000 0x560f9bc91700 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.540 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.531+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v35'30 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.531+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560f9bdf5800 0x560f9be8a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.531+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf5c00 con 0x560f9bdf5800 2024-09-13T23:43:50.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.531+0000 7fd8c8374700 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560f9bdf5800 0x560f9be8a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.531+0000 7fd8c8374700 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560f9bdf5800 0x560f9be8a580 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.535+0000 7fd8c8374700 1 -- 172.21.15.167:0/2227567392 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v35'18 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdf5800 2024-09-13T23:43:50.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.535+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560f9be84400 0x560f9be8ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.535+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be84800 con 0x560f9be84400 2024-09-13T23:43:50.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.535+0000 7fd8c9376700 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560f9be84400 0x560f9be8ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.535+0000 7fd8c9376700 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560f9be84400 0x560f9be8ab00 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.539+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v35'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.539+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560f9be84c00 0x560f9be8b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.539+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85000 con 0x560f9be84c00 2024-09-13T23:43:50.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.539+0000 7fd8c8b75700 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560f9be84c00 0x560f9be8b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.539+0000 7fd8c8b75700 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560f9be84c00 0x560f9be8b080 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.539+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v35'153 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.539+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85400 con 0x560f9bdf5000 2024-09-13T23:43:50.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.543+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v35'44 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.543+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85800 con 0x560f9be84400 2024-09-13T23:43:50.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.547+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v35'32 uv10 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.547+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85c00 con 0x560f9bdf5000 2024-09-13T23:43:50.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.547+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v35'20 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.547+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf5400 con 0x560f9be84c00 2024-09-13T23:43:50.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.551+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v35'30 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.551+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf400 con 0x560f9bdf5000 2024-09-13T23:43:50.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.555+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v35'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.555+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf800 con 0x560f9bdf5000 2024-09-13T23:43:50.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.555+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v35'25 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.555+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9bdf5000 2024-09-13T23:43:50.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.559+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v35'45 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.559+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd44800 con 0x560f9be84400 2024-09-13T23:43:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.559+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v35'21 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.559+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd45c00 con 0x560f9bdf5000 2024-09-13T23:43:50.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.563+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v35'18 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.563+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85000 con 0x560f9be84c00 2024-09-13T23:43:50.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.567+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v35'35 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.567+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85400 con 0x560f9be84c00 2024-09-13T23:43:50.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.567+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v35'154 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.567+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85c00 con 0x560f9bdf5000 2024-09-13T23:43:50.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.571+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v35'38 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.571+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be84800 con 0x560f9bdf5000 2024-09-13T23:43:50.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.575+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v35'46 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.575+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf5400 con 0x560f9be84400 2024-09-13T23:43:50.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.579+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v35'19 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.579+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4000 con 0x560f9bdf5000 2024-09-13T23:43:50.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.579+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v35'26 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.579+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4400 con 0x560f9be84c00 2024-09-13T23:43:50.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.583+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v35'155 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.583+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf400 con 0x560f9bdf5000 2024-09-13T23:43:50.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.587+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v35'39 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.587+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf800 con 0x560f9bdf5000 2024-09-13T23:43:50.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.591+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v35'28 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.591+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdce800 con 0x560f9be84400 2024-09-13T23:43:50.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.591+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v35'20 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.591+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9bdf5000 2024-09-13T23:43:50.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.595+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v35'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.595+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9be84400 2024-09-13T23:43:50.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.599+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v35'14 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.599+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf400 con 0x560f9be84400 2024-09-13T23:43:50.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.599+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v35'33 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.599+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4000 con 0x560f9be84400 2024-09-13T23:43:50.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.603+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v35'20 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.603+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4400 con 0x560f9bdf5000 2024-09-13T23:43:50.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.607+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v35'18 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.607+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85000 con 0x560f9be84c00 2024-09-13T23:43:50.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.607+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v35'31 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.611+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85400 con 0x560f9bdf5000 2024-09-13T23:43:50.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.611+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v35'40 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.611+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85c00 con 0x560f9be84c00 2024-09-13T23:43:50.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.615+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v35'36 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.615+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be84800 con 0x560f9bdf5800 2024-09-13T23:43:50.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.619+0000 7fd8c8374700 1 -- 172.21.15.167:0/2227567392 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v35'24 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdf5800 2024-09-13T23:43:50.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.619+0000 7fd8ab33a700 20 reqs_thread_entry: start 2024-09-13T23:43:50.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.619+0000 7fd8ca3bcf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:50.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.619+0000 7fd8ca3bcf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:50.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.619+0000 7fd8a9336700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:20 2024 2024-09-13T23:43:50.630 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:50.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.619+0000 7fd8a8b14700 20 reqs_thread_entry: start 2024-09-13T23:43:50.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.619+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9bdf5000 2024-09-13T23:43:50.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.623+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(33 gc.0 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.623+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf400 con 0x560f9bdf5000 2024-09-13T23:43:50.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.623+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(34 gc.0 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.623+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:35 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf5400 con 0x560f9bdf5000 2024-09-13T23:43:50.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.623+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(35 gc.0 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.623+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:36 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd44800 con 0x560f9bdf5800 2024-09-13T23:43:50.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.623+0000 7fd8c8374700 1 -- 172.21.15.167:0/2227567392 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(36 gc.1 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdf5800 2024-09-13T23:43:50.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.623+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:37 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85800 con 0x560f9bdf5800 2024-09-13T23:43:50.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.623+0000 7fd8c8374700 1 -- 172.21.15.167:0/2227567392 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(37 gc.1 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdf5800 2024-09-13T23:43:50.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.627+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:38 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf5c00 con 0x560f9bdf5800 2024-09-13T23:43:50.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.627+0000 7fd8c8374700 1 -- 172.21.15.167:0/2227567392 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(38 gc.1 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdf5800 2024-09-13T23:43:50.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.627+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:39 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4400 con 0x560f9be84400 2024-09-13T23:43:50.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.627+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(39 gc.2 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.627+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:40 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf5400 con 0x560f9be84400 2024-09-13T23:43:50.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.627+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(40 gc.2 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.627+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:41 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf800 con 0x560f9be84400 2024-09-13T23:43:50.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.627+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(41 gc.2 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.627+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:42 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9be84c00 2024-09-13T23:43:50.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.627+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(42 gc.3 [call out=15b] v0'0 uv15 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.627+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:43 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9be84c00 2024-09-13T23:43:50.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.631+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(43 gc.3 [call out=24b] v0'0 uv15 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.631+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:44 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf400 con 0x560f9be84c00 2024-09-13T23:43:50.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.631+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(44 gc.3 [call out=15b] v0'0 uv15 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.631+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be84800 con 0x560f9bdf5000 2024-09-13T23:43:50.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.631+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(45 gc.4 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.631+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:46 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85800 con 0x560f9bdf5000 2024-09-13T23:43:50.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.631+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(46 gc.4 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.631+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:47 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85000 con 0x560f9bdf5000 2024-09-13T23:43:50.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.631+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(47 gc.4 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.631+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:48 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85400 con 0x560f9be84400 2024-09-13T23:43:50.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.635+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(48 gc.5 [call out=15b] v0'0 uv10 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.635+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:49 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85c00 con 0x560f9be84400 2024-09-13T23:43:50.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.635+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(49 gc.5 [call out=24b] v0'0 uv10 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.635+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:50 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd44800 con 0x560f9be84400 2024-09-13T23:43:50.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.635+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(50 gc.5 [call out=15b] v0'0 uv10 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.635+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:51 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd44800 con 0x560f9bdf5000 2024-09-13T23:43:50.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.635+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(51 gc.6 [call out=15b] v0'0 uv6 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.635+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85400 con 0x560f9bdf5000 2024-09-13T23:43:50.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.639+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(52 gc.6 [call out=24b] v0'0 uv6 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.639+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85c00 con 0x560f9bdf5000 2024-09-13T23:43:50.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.639+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(53 gc.6 [call out=15b] v0'0 uv6 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.639+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:54 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf800 con 0x560f9be84c00 2024-09-13T23:43:50.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.639+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(54 gc.7 [call out=15b] v0'0 uv8 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.639+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:55 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4000 con 0x560f9be84c00 2024-09-13T23:43:50.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.639+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(55 gc.7 [call out=24b] v0'0 uv8 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.639+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:56 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4400 con 0x560f9be84c00 2024-09-13T23:43:50.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.639+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(56 gc.7 [call out=15b] v0'0 uv8 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.639+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:57 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf5400 con 0x560f9bdf5000 2024-09-13T23:43:50.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.639+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(57 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.639+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:58 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9bdf5000 2024-09-13T23:43:50.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.643+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(58 gc.8 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.643+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:59 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9bdf5000 2024-09-13T23:43:50.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.643+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(59 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.643+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:60 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf400 con 0x560f9bdf5000 2024-09-13T23:43:50.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.643+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(60 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.643+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:61 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf800 con 0x560f9bdf5000 2024-09-13T23:43:50.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.643+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(61 gc.9 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.643+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:62 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be84800 con 0x560f9bdf5000 2024-09-13T23:43:50.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.647+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(62 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.647+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85800 con 0x560f9bdf5000 2024-09-13T23:43:50.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.647+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 31 ==== osd_op_reply(63 gc.10 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.647+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85000 con 0x560f9bdf5000 2024-09-13T23:43:50.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.647+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 32 ==== osd_op_reply(64 gc.10 [call out=24b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.647+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85400 con 0x560f9bdf5000 2024-09-13T23:43:50.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.647+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 33 ==== osd_op_reply(65 gc.10 [call out=15b] v0'0 uv13 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.647+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:66 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85c00 con 0x560f9be84400 2024-09-13T23:43:50.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.651+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(66 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.651+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd44800 con 0x560f9be84400 2024-09-13T23:43:50.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.651+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(67 gc.11 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.651+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be84800 con 0x560f9be84400 2024-09-13T23:43:50.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.651+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(68 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.651+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85800 con 0x560f9bdf5000 2024-09-13T23:43:50.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.651+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 34 ==== osd_op_reply(69 gc.12 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.651+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85000 con 0x560f9bdf5000 2024-09-13T23:43:50.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.651+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 35 ==== osd_op_reply(70 gc.12 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.651+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85400 con 0x560f9bdf5000 2024-09-13T23:43:50.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.655+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 36 ==== osd_op_reply(71 gc.12 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.655+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9be84c00 2024-09-13T23:43:50.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.655+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(72 gc.13 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.655+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9be84c00 2024-09-13T23:43:50.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.655+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(73 gc.13 [call out=24b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.655+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:74 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf400 con 0x560f9be84c00 2024-09-13T23:43:50.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.655+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(74 gc.13 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.655+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:75 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf800 con 0x560f9be84c00 2024-09-13T23:43:50.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.655+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(75 gc.14 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.659+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4000 con 0x560f9be84c00 2024-09-13T23:43:50.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.659+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(76 gc.14 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.659+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:77 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4400 con 0x560f9be84c00 2024-09-13T23:43:50.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.659+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(77 gc.14 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.659+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:78 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf5400 con 0x560f9bdf5000 2024-09-13T23:43:50.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.659+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 37 ==== osd_op_reply(78 gc.15 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.659+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:79 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2e000 con 0x560f9bdf5000 2024-09-13T23:43:50.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.659+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 38 ==== osd_op_reply(79 gc.15 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.659+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2e400 con 0x560f9bdf5000 2024-09-13T23:43:50.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.663+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 39 ==== osd_op_reply(80 gc.15 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.663+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:81 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2e800 con 0x560f9bdf5000 2024-09-13T23:43:50.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.663+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 40 ==== osd_op_reply(81 gc.16 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.663+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:82 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2ec00 con 0x560f9bdf5000 2024-09-13T23:43:50.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.663+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 41 ==== osd_op_reply(82 gc.16 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.663+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:83 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2f000 con 0x560f9bdf5000 2024-09-13T23:43:50.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.663+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 42 ==== osd_op_reply(83 gc.16 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.663+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:84 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2f400 con 0x560f9be84400 2024-09-13T23:43:50.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.667+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(84 gc.17 [call out=15b] v0'0 uv18 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.667+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:85 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2f800 con 0x560f9be84400 2024-09-13T23:43:50.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.667+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(85 gc.17 [call out=24b] v0'0 uv18 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.667+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:86 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2fc00 con 0x560f9be84400 2024-09-13T23:43:50.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.667+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(86 gc.17 [call out=1700b] v0'0 uv18 ondisk = 0) v8 ==== 149+0+1700 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.667+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:87 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2e000 con 0x560f9bdf5000 2024-09-13T23:43:50.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.667+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 43 ==== osd_op_reply(87 gc.18 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.667+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:88 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2e400 con 0x560f9bdf5000 2024-09-13T23:43:50.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.671+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 44 ==== osd_op_reply(88 gc.18 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.671+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:89 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4000 con 0x560f9bdf5000 2024-09-13T23:43:50.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.671+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 45 ==== osd_op_reply(89 gc.18 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.671+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:90 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4400 con 0x560f9be84c00 2024-09-13T23:43:50.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.671+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(90 gc.19 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.671+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:91 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf5400 con 0x560f9be84c00 2024-09-13T23:43:50.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.671+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(91 gc.19 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.671+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:92 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9be84c00 2024-09-13T23:43:50.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.671+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(92 gc.19 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.671+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:93 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9bdf5000 2024-09-13T23:43:50.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.675+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 46 ==== osd_op_reply(93 gc.20 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.675+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:94 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf400 con 0x560f9bdf5000 2024-09-13T23:43:50.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.675+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 47 ==== osd_op_reply(94 gc.20 [call out=24b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.675+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:95 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf800 con 0x560f9bdf5000 2024-09-13T23:43:50.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.675+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 48 ==== osd_op_reply(95 gc.20 [call out=15b] v0'0 uv9 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.675+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:96 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85800 con 0x560f9bdf5000 2024-09-13T23:43:50.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.675+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 49 ==== osd_op_reply(96 gc.21 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.675+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:97 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85000 con 0x560f9bdf5000 2024-09-13T23:43:50.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.679+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 50 ==== osd_op_reply(97 gc.21 [call out=24b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.679+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:98 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85400 con 0x560f9bdf5000 2024-09-13T23:43:50.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.679+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 51 ==== osd_op_reply(98 gc.21 [call out=15b] v0'0 uv7 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.679+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:99 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85800 con 0x560f9be84400 2024-09-13T23:43:50.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.679+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(99 gc.22 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.679+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:100 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9be84400 2024-09-13T23:43:50.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.679+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(100 gc.22 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.679+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:101 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9be84400 2024-09-13T23:43:50.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.683+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(101 gc.22 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.683+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:102 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf400 con 0x560f9bdf5000 2024-09-13T23:43:50.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.683+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 52 ==== osd_op_reply(102 gc.23 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.683+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:103 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf800 con 0x560f9bdf5000 2024-09-13T23:43:50.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.683+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 53 ==== osd_op_reply(103 gc.23 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.683+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:104 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4000 con 0x560f9bdf5000 2024-09-13T23:43:50.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.683+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 54 ==== osd_op_reply(104 gc.23 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.687+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:105 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4400 con 0x560f9be84400 2024-09-13T23:43:50.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.687+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== osd_op_reply(105 gc.24 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.687+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:106 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf5400 con 0x560f9be84400 2024-09-13T23:43:50.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.687+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 25 ==== osd_op_reply(106 gc.24 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.687+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:107 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2e800 con 0x560f9be84400 2024-09-13T23:43:50.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.687+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 26 ==== osd_op_reply(107 gc.24 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.687+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:108 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2ec00 con 0x560f9be84400 2024-09-13T23:43:50.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.687+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 27 ==== osd_op_reply(108 gc.25 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.687+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:109 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2f000 con 0x560f9be84400 2024-09-13T23:43:50.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.687+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 28 ==== osd_op_reply(109 gc.25 [call out=24b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.687+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:110 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2e000 con 0x560f9be84400 2024-09-13T23:43:50.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.691+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 29 ==== osd_op_reply(110 gc.25 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.691+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:111 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2e400 con 0x560f9be84400 2024-09-13T23:43:50.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.691+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 30 ==== osd_op_reply(111 gc.26 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.691+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:112 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9be84400 2024-09-13T23:43:50.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.691+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 31 ==== osd_op_reply(112 gc.26 [call out=24b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.691+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:113 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85c00 con 0x560f9be84400 2024-09-13T23:43:50.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.691+0000 7fd8c9376700 1 -- 172.21.15.167:0/2227567392 <== osd.3 v2:172.21.15.167:6816/3688363524 32 ==== osd_op_reply(113 gc.26 [call out=15b] v0'0 uv3 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9be84400 2024-09-13T23:43:50.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.691+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:114 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be84800 con 0x560f9bdf5000 2024-09-13T23:43:50.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.691+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 55 ==== osd_op_reply(114 gc.27 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.691+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:115 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9be85800 con 0x560f9bdf5000 2024-09-13T23:43:50.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.695+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 56 ==== osd_op_reply(115 gc.27 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.695+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:116 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2f400 con 0x560f9bdf5000 2024-09-13T23:43:50.704 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.695+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 57 ==== osd_op_reply(116 gc.27 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.695+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:117 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2f800 con 0x560f9be84c00 2024-09-13T23:43:50.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.695+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(117 gc.28 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.695+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:118 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2fc00 con 0x560f9be84c00 2024-09-13T23:43:50.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.695+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(118 gc.28 [call out=24b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.695+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:119 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd44800 con 0x560f9be84c00 2024-09-13T23:43:50.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.699+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(119 gc.28 [call out=15b] v0'0 uv4 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.699+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:120 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9bdf5000 2024-09-13T23:43:50.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.699+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 58 ==== osd_op_reply(120 gc.29 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.708 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.699+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:121 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9bdf5000 2024-09-13T23:43:50.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.699+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 59 ==== osd_op_reply(121 gc.29 [call out=24b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.699+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:122 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2e800 con 0x560f9bdf5000 2024-09-13T23:43:50.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.699+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.0 v2:172.21.15.12:6800/2649399468 60 ==== osd_op_reply(122 gc.29 [call out=15b] v0'0 uv11 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bdf5000 2024-09-13T23:43:50.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.699+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:123 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2ec00 con 0x560f9be84c00 2024-09-13T23:43:50.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.703+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(123 gc.30 [call out=15b] v0'0 uv8 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.703+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:124 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2f000 con 0x560f9be84c00 2024-09-13T23:43:50.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.703+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(124 gc.30 [call out=24b] v0'0 uv8 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.703+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:125 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2e000 con 0x560f9be84c00 2024-09-13T23:43:50.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.703+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 <== osd.1 v2:172.21.15.167:6800/3875661733 28 ==== osd_op_reply(125 gc.30 [call out=15b] v0'0 uv8 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9be84c00 2024-09-13T23:43:50.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.703+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:126 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2e400 con 0x560f9bdf5800 2024-09-13T23:43:50.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.703+0000 7fd8c8374700 1 -- 172.21.15.167:0/2227567392 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(126 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdf5800 2024-09-13T23:43:50.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.703+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:127 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf4400 con 0x560f9bdf5800 2024-09-13T23:43:50.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.703+0000 7fd8c8374700 1 -- 172.21.15.167:0/2227567392 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(127 gc.31 [call out=24b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdf5800 2024-09-13T23:43:50.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.703+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:128 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdf5400 con 0x560f9bdf5800 2024-09-13T23:43:50.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.707+0000 7fd8c8374700 1 -- 172.21.15.167:0/2227567392 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(128 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdf5800 2024-09-13T23:43:50.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.707+0000 7fd8afb43700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:50.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.707+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94625039613952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2f400 con 0x560f9bd44400 2024-09-13T23:43:50.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.711+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 data_loggenerations_metadata [watch unwatch cookie 94625039613952] v35'156 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.711+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94625038097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2f800 con 0x560f9bd44c00 2024-09-13T23:43:50.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.711+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(63 notify.0 [watch unwatch cookie 94625038097152] v35'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.711+0000 7fd8ca3bcf40 20 remove_watcher() i=0 2024-09-13T23:43:50.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.711+0000 7fd8ca3bcf40 2 removed watcher, disabling cache 2024-09-13T23:43:50.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.711+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94625038098304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2fc00 con 0x560f9bd44400 2024-09-13T23:43:50.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.715+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 notify.1 [watch unwatch cookie 94625038098304] v35'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bd44400 2024-09-13T23:43:50.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.715+0000 7fd8ca3bcf40 20 remove_watcher() i=1 2024-09-13T23:43:50.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.715+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94625038099456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bf2e800 con 0x560f9bd44c00 2024-09-13T23:43:50.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.719+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 notify.2 [watch unwatch cookie 94625038099456] v35'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.719+0000 7fd8ca3bcf40 20 remove_watcher() i=2 2024-09-13T23:43:50.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.719+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94625038100608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf400 con 0x560f9bdce000 2024-09-13T23:43:50.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.719+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(66 notify.3 [watch unwatch cookie 94625038100608] v35'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.719+0000 7fd8ca3bcf40 20 remove_watcher() i=3 2024-09-13T23:43:50.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.719+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94625038101760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcf800 con 0x560f9bd44c00 2024-09-13T23:43:50.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.723+0000 7fd8c8b75700 1 -- 172.21.15.167:0/1358080846 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(67 notify.4 [watch unwatch cookie 94625038101760] v35'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d200 con 0x560f9bd44c00 2024-09-13T23:43:50.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.723+0000 7fd8ca3bcf40 20 remove_watcher() i=4 2024-09-13T23:43:50.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.723+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94625038102912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcec00 con 0x560f9bdce400 2024-09-13T23:43:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.727+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(68 notify.5 [watch unwatch cookie 94625038102912] v35'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.727+0000 7fd8ca3bcf40 20 remove_watcher() i=5 2024-09-13T23:43:50.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.727+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94625038104064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bdcfc00 con 0x560f9bdce400 2024-09-13T23:43:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.727+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(69 notify.6 [watch unwatch cookie 94625038104064] v35'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9d440 con 0x560f9bdce400 2024-09-13T23:43:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.727+0000 7fd8ca3bcf40 20 remove_watcher() i=6 2024-09-13T23:43:50.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.727+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94625038105216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x560f9bd44800 con 0x560f9bdce000 2024-09-13T23:43:50.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8c9376700 1 -- 172.21.15.167:0/1358080846 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(70 notify.7 [watch unwatch cookie 94625038105216] v35'161 uv159 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x560f9bd9cb40 con 0x560f9bdce000 2024-09-13T23:43:50.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 20 remove_watcher() i=7 2024-09-13T23:43:50.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560f9bd44c00 msgr2=0x560f9bc90100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560f9bd44c00 0x560f9bc90100 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560f9bd44400 msgr2=0x560f9bc8fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560f9bd44400 0x560f9bc8fb80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560f9bdce000 msgr2=0x560f9bc8e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560f9bdce000 0x560f9bc8e580 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560f9bdce400 msgr2=0x560f9bc90c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560f9bdce400 0x560f9bc90c00 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9bd45800 msgr2=0x560f9bc90680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9bd45800 0x560f9bc90680 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x560f9bd67d10 tx=0x560f9bddc000 comp rx=0 tx=0).stop 2024-09-13T23:43:50.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 msgr2=0x560f9bc8f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8c8374700 1 -- 172.21.15.167:0/1358080846 reap_dead start 2024-09-13T23:43:50.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 0x560f9bc8f600 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x560f9c81c6c0 tx=0x560f9bd0ce40 comp rx=0 tx=0).stop 2024-09-13T23:43:50.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 shutdown_connections 2024-09-13T23:43:50.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/1358080846 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bd44000 0x560f9bc8f600 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 >> 172.21.15.167:0/1358080846 conn(0x560f9b9a9800 msgr2=0x560f9baa9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:50.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 shutdown_connections 2024-09-13T23:43:50.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/1358080846 wait complete. 2024-09-13T23:43:50.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560f9bdf5000 msgr2=0x560f9bc91700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x560f9bdf5000 0x560f9bc91700 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560f9be84c00 msgr2=0x560f9be8b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x560f9be84c00 0x560f9be8b080 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560f9bdf5800 msgr2=0x560f9be8a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.731+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x560f9bdf5800 0x560f9be8a580 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560f9be84400 msgr2=0x560f9be8ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x560f9be84400 0x560f9be8ab00 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9be84000 msgr2=0x560f9be8a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9be84000 0x560f9be8a000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x560f9bd8fb00 tx=0x560f9bd0ca80 comp rx=0 tx=0).stop 2024-09-13T23:43:50.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 msgr2=0x560f9bc91180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8c8b75700 1 -- 172.21.15.167:0/2227567392 reap_dead start 2024-09-13T23:43:50.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 0x560f9bc91180 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x560f9be00d50 tx=0x560f9bddc0c0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 shutdown_connections 2024-09-13T23:43:50.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2227567392 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9bdf4c00 0x560f9bc91180 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 >> 172.21.15.167:0/2227567392 conn(0x560f9bdf4800 msgr2=0x560f9baaa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:50.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 shutdown_connections 2024-09-13T23:43:50.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2227567392 wait complete. 2024-09-13T23:43:50.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2776512916 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9b9a9c00 msgr2=0x560f9bc8f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2776512916 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9b9a9c00 0x560f9bc8f080 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x560f9c81c8d0 tx=0x560f9bd0cb40 comp rx=0 tx=0).stop 2024-09-13T23:43:50.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2776512916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 msgr2=0x560f9bc8eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2776512916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 0x560f9bc8eb00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x560f9c80df80 tx=0x560f9baad920 comp rx=0 tx=0).stop 2024-09-13T23:43:50.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2776512916 shutdown_connections 2024-09-13T23:43:50.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2776512916 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x560f9b9a9c00 0x560f9bc8f080 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 --2- 172.21.15.167:0/2776512916 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x560f9b9a9400 0x560f9bc8eb00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2776512916 >> 172.21.15.167:0/2776512916 conn(0x560f9b9a8c00 msgr2=0x560f9baa9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:50.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2776512916 shutdown_connections 2024-09-13T23:43:50.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.735+0000 7fd8ca3bcf40 1 -- 172.21.15.167:0/2776512916 wait complete. 2024-09-13T23:43:50.749 INFO:teuthology.orchestra.run.smithi167.stdout:[ 2024-09-13T23:43:50.749 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:50.750 INFO:teuthology.orchestra.run.smithi167.stdout: "tag": "9144208c-3c85-4fb6-9007-2693d5a930f0.4222.959130024862934607\u0000", 2024-09-13T23:43:50.750 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:43:50.392564+0000", 2024-09-13T23:43:50.750 INFO:teuthology.orchestra.run.smithi167.stdout: "objs": [ 2024-09-13T23:43:50.750 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:50.750 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": "default.rgw.buckets.data", 2024-09-13T23:43:50.750 INFO:teuthology.orchestra.run.smithi167.stdout: "oid": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_1", 2024-09-13T23:43:50.750 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "", 2024-09-13T23:43:50.750 INFO:teuthology.orchestra.run.smithi167.stdout: "instance": "" 2024-09-13T23:43:50.750 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:50.750 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:50.750 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": "default.rgw.buckets.data", 2024-09-13T23:43:50.750 INFO:teuthology.orchestra.run.smithi167.stdout: "oid": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_2", 2024-09-13T23:43:50.751 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "", 2024-09-13T23:43:50.751 INFO:teuthology.orchestra.run.smithi167.stdout: "instance": "" 2024-09-13T23:43:50.751 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:50.751 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:50.751 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": "default.rgw.buckets.data", 2024-09-13T23:43:50.751 INFO:teuthology.orchestra.run.smithi167.stdout: "oid": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_3", 2024-09-13T23:43:50.751 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "", 2024-09-13T23:43:50.751 INFO:teuthology.orchestra.run.smithi167.stdout: "instance": "" 2024-09-13T23:43:50.751 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:50.751 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:50.751 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": "default.rgw.buckets.data", 2024-09-13T23:43:50.751 INFO:teuthology.orchestra.run.smithi167.stdout: "oid": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_4", 2024-09-13T23:43:50.752 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "", 2024-09-13T23:43:50.752 INFO:teuthology.orchestra.run.smithi167.stdout: "instance": "" 2024-09-13T23:43:50.752 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:50.752 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:50.752 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": "default.rgw.buckets.data", 2024-09-13T23:43:50.752 INFO:teuthology.orchestra.run.smithi167.stdout: "oid": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_5", 2024-09-13T23:43:50.752 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "", 2024-09-13T23:43:50.752 INFO:teuthology.orchestra.run.smithi167.stdout: "instance": "" 2024-09-13T23:43:50.752 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:50.752 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:50.752 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": "default.rgw.buckets.data", 2024-09-13T23:43:50.753 INFO:teuthology.orchestra.run.smithi167.stdout: "oid": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_6", 2024-09-13T23:43:50.753 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "", 2024-09-13T23:43:50.753 INFO:teuthology.orchestra.run.smithi167.stdout: "instance": "" 2024-09-13T23:43:50.753 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:50.753 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:50.753 INFO:teuthology.orchestra.run.smithi167.stdout: "pool": "default.rgw.buckets.data", 2024-09-13T23:43:50.753 INFO:teuthology.orchestra.run.smithi167.stdout: "oid": "9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_7", 2024-09-13T23:43:50.753 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "", 2024-09-13T23:43:50.753 INFO:teuthology.orchestra.run.smithi167.stdout: "instance": "" 2024-09-13T23:43:50.753 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:50.753 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:43:50.754 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:50.754 INFO:teuthology.orchestra.run.smithi167.stdout:] 2024-09-13T23:43:50.754 DEBUG:tasks.util.rgw: json result: [{'tag': '9144208c-3c85-4fb6-9007-2693d5a930f0.4222.959130024862934607\x00', 'time': '2024-09-13T23:43:50.392564+0000', 'objs': [{'pool': 'default.rgw.buckets.data', 'oid': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_1', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_2', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_3', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_4', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_5', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_6', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_7', 'key': '', 'instance': ''}]}] 2024-09-13T23:43:50.754 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process'] 2024-09-13T23:43:50.754 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process'] 2024-09-13T23:43:50.754 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process 2024-09-13T23:43:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:50.832 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.823+0000 7f0578050f40 1 Processor -- start 2024-09-13T23:43:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.823+0000 7f0578050f40 1 -- start start 2024-09-13T23:43:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.823+0000 7f0578050f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x563929bb1000 0x563929e96580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.823+0000 7f0578050f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x563929c75500 con 0x563929bb1400 2024-09-13T23:43:50.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.823+0000 7f0578050f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x563929c3ea00 con 0x563929bb1000 2024-09-13T23:43:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.823+0000 7f0576809700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x563929bb1000 0x563929e96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.823+0000 7f0576008700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x563929bb1400 0x563929cb0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:49310/0 (socket says 172.21.15.167:49310) 2024-09-13T23:43:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.823+0000 7f0576008700 1 -- 172.21.15.167:0/701027985 learned_addr learned my addr 172.21.15.167:0/701027985 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.823+0000 7f0576809700 1 -- 172.21.15.167:0/701027985 >> v1:172.21.15.12:6789/0 conn(0x563929bb1400 legacy=0x563929cb0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.823+0000 7f0575807700 1 -- 172.21.15.167:0/701027985 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848826765 0 0) 0x563929d0c000 con 0x563929bb1400 2024-09-13T23:43:50.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.823+0000 7f0576809700 1 -- 172.21.15.167:0/701027985 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563929bc3d40 con 0x563929bb1000 2024-09-13T23:43:50.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0576809700 1 --2- 172.21.15.167:0/701027985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x563929e91a70 tx=0x563929cb5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c073ff20fd212d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0575807700 1 -- 172.21.15.167:0/701027985 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563929c554a0 con 0x563929bb1000 2024-09-13T23:43:50.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0575807700 1 -- 172.21.15.167:0/701027985 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563929c75800 con 0x563929bb1000 2024-09-13T23:43:50.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0575807700 1 -- 172.21.15.167:0/701027985 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563929c55600 con 0x563929bb1000 2024-09-13T23:43:50.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0578050f40 1 -- 172.21.15.167:0/701027985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 msgr2=0x563929e96580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0578050f40 1 --2- 172.21.15.167:0/701027985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x563929e91a70 tx=0x563929cb5920 comp rx=0 tx=0).stop 2024-09-13T23:43:50.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0578050f40 1 -- 172.21.15.167:0/701027985 shutdown_connections 2024-09-13T23:43:50.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0578050f40 1 --2- 172.21.15.167:0/701027985 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0578050f40 1 -- 172.21.15.167:0/701027985 >> 172.21.15.167:0/701027985 conn(0x563929bb0c00 msgr2=0x563929cb0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:50.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0578050f40 1 -- 172.21.15.167:0/701027985 shutdown_connections 2024-09-13T23:43:50.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0578050f40 1 -- 172.21.15.167:0/701027985 wait complete. 2024-09-13T23:43:50.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0578050f40 1 Processor -- start 2024-09-13T23:43:50.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0578050f40 1 -- start start 2024-09-13T23:43:50.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0578050f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0578050f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563929c3e500 con 0x563929bb1000 2024-09-13T23:43:50.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0576809700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0576809700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52142/0 (socket says 172.21.15.167:52142) 2024-09-13T23:43:50.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0576809700 1 -- 172.21.15.167:0/4214327744 learned_addr learned my addr 172.21.15.167:0/4214327744 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:50.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.827+0000 7f0576809700 1 -- 172.21.15.167:0/4214327744 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563929bc3ba0 con 0x563929bb1000 2024-09-13T23:43:50.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0576809700 1 --2- 172.21.15.167:0/4214327744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x56392a985e00 tx=0x563929cb5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f5b0c2f47a9f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0575006700 1 -- 172.21.15.167:0/4214327744 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563929c55760 con 0x563929bb1000 2024-09-13T23:43:50.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0575006700 1 -- 172.21.15.167:0/4214327744 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563929c75b00 con 0x563929bb1000 2024-09-13T23:43:50.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0575006700 1 -- 172.21.15.167:0/4214327744 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563929c551e0 con 0x563929bb1000 2024-09-13T23:43:50.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0578050f40 1 -- 172.21.15.167:0/4214327744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 msgr2=0x563929e96580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0578050f40 1 --2- 172.21.15.167:0/4214327744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x56392a985e00 tx=0x563929cb5800 comp rx=0 tx=0).stop 2024-09-13T23:43:50.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0578050f40 1 -- 172.21.15.167:0/4214327744 shutdown_connections 2024-09-13T23:43:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0578050f40 1 --2- 172.21.15.167:0/4214327744 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0578050f40 1 -- 172.21.15.167:0/4214327744 >> 172.21.15.167:0/4214327744 conn(0x563929bb0c00 msgr2=0x563929cb1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0578050f40 1 -- 172.21.15.167:0/4214327744 shutdown_connections 2024-09-13T23:43:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0578050f40 1 -- 172.21.15.167:0/4214327744 wait complete. 2024-09-13T23:43:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0578050f40 1 Processor -- start 2024-09-13T23:43:50.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0578050f40 1 -- start start 2024-09-13T23:43:50.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0578050f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0578050f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563929c3e3c0 con 0x563929bb1000 2024-09-13T23:43:50.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0576809700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0576809700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52152/0 (socket says 172.21.15.167:52152) 2024-09-13T23:43:50.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0576809700 1 -- 172.21.15.167:0/1612550526 learned_addr learned my addr 172.21.15.167:0/1612550526 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:50.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0576809700 1 -- 172.21.15.167:0/1612550526 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563929bc3a00 con 0x563929bb1000 2024-09-13T23:43:50.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.831+0000 7f0576809700 1 --2- 172.21.15.167:0/1612550526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x56392a985ef0 tx=0x563929cb5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0573803700 1 -- 172.21.15.167:0/1612550526 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563929c554a0 con 0x563929bb1000 2024-09-13T23:43:50.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0573803700 1 -- 172.21.15.167:0/1612550526 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563929c75c80 con 0x563929bb1000 2024-09-13T23:43:50.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0578050f40 1 -- 172.21.15.167:0/1612550526 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563929bc3860 con 0x563929bb1000 2024-09-13T23:43:50.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0573803700 1 -- 172.21.15.167:0/1612550526 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563929c55600 con 0x563929bb1000 2024-09-13T23:43:50.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0578050f40 1 -- 172.21.15.167:0/1612550526 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563929bc36c0 con 0x563929bb1000 2024-09-13T23:43:50.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0578050f40 1 Processor -- start 2024-09-13T23:43:50.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0578050f40 1 -- start start 2024-09-13T23:43:50.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0578050f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 0x563929e96b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0578050f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563929bd6640 con 0x563929bb1800 2024-09-13T23:43:50.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f057700a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 0x563929e96b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f057700a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 0x563929e96b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52162/0 (socket says 172.21.15.167:52162) 2024-09-13T23:43:50.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f057700a700 1 -- 172.21.15.167:0/3281459177 learned_addr learned my addr 172.21.15.167:0/3281459177 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:50.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0573803700 1 -- 172.21.15.167:0/1612550526 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x563929caab00 con 0x563929bb1000 2024-09-13T23:43:50.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f057700a700 1 -- 172.21.15.167:0/3281459177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563929bc2000 con 0x563929bb1800 2024-09-13T23:43:50.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0573803700 1 --2- 172.21.15.167:0/1612550526 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563929f6c000 0x563929e97080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0573803700 1 -- 172.21.15.167:0/1612550526 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x563929c17180 con 0x563929bb1000 2024-09-13T23:43:50.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0576809700 1 --2- 172.21.15.167:0/1612550526 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563929f6c000 0x563929e97080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f057700a700 1 --2- 172.21.15.167:0/3281459177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 0x563929e96b00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x563929ecd380 tx=0x563929f228a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d5f4192dc31843 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0570ffe700 1 -- 172.21.15.167:0/3281459177 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563929c554a0 con 0x563929bb1800 2024-09-13T23:43:50.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0570ffe700 1 -- 172.21.15.167:0/3281459177 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563929f20000 con 0x563929bb1800 2024-09-13T23:43:50.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.835+0000 7f0570ffe700 1 -- 172.21.15.167:0/3281459177 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563929c55600 con 0x563929bb1800 2024-09-13T23:43:50.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f0578050f40 1 -- 172.21.15.167:0/3281459177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 msgr2=0x563929e96b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f0578050f40 1 --2- 172.21.15.167:0/3281459177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 0x563929e96b00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x563929ecd380 tx=0x563929f228a0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f0576809700 1 --2- 172.21.15.167:0/1612550526 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563929f6c000 0x563929e97080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x563929ecd680 tx=0x563929f22120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f0578050f40 1 -- 172.21.15.167:0/3281459177 shutdown_connections 2024-09-13T23:43:50.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f0578050f40 1 --2- 172.21.15.167:0/3281459177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 0x563929e96b00 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f0578050f40 1 -- 172.21.15.167:0/3281459177 >> 172.21.15.167:0/3281459177 conn(0x563929bb1400 msgr2=0x563929cb1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:50.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f0578050f40 1 -- 172.21.15.167:0/3281459177 shutdown_connections 2024-09-13T23:43:50.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f0578050f40 1 -- 172.21.15.167:0/3281459177 wait complete. 2024-09-13T23:43:50.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f0578050f40 1 Processor -- start 2024-09-13T23:43:50.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f0578050f40 1 -- start start 2024-09-13T23:43:50.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f0578050f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 0x563929e96b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f0578050f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563929bd68c0 con 0x563929bb1800 2024-09-13T23:43:50.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f057700a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 0x563929e96b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f057700a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 0x563929e96b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52166/0 (socket says 172.21.15.167:52166) 2024-09-13T23:43:50.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 learned_addr learned my addr 172.21.15.167:0/4192659245 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:50.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563929bc2ea0 con 0x563929bb1800 2024-09-13T23:43:50.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.839+0000 7f057700a700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 0x563929e96b00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x563929eccc00 tx=0x563929cb5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f056f7fb700 1 -- 172.21.15.167:0/4192659245 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563929c55760 con 0x563929bb1800 2024-09-13T23:43:50.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f056f7fb700 1 -- 172.21.15.167:0/4192659245 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563929f20180 con 0x563929bb1800 2024-09-13T23:43:50.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563929bc2820 con 0x563929bb1800 2024-09-13T23:43:50.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f056f7fb700 1 -- 172.21.15.167:0/4192659245 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563929c554a0 con 0x563929bb1800 2024-09-13T23:43:50.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563929bc29c0 con 0x563929bb1800 2024-09-13T23:43:50.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f056f7fb700 1 -- 172.21.15.167:0/4192659245 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x563929caadc0 con 0x563929bb1800 2024-09-13T23:43:50.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f0578050f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f056f7fb700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563929f6cc00 0x563929e97600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f056f7fb700 1 -- 172.21.15.167:0/4192659245 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x563929c17340 con 0x563929bb1800 2024-09-13T23:43:50.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f0578050f40 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563929f6d000 0x563929e97b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f0576008700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563929f6cc00 0x563929e97600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d400 con 0x563929f6d000 2024-09-13T23:43:50.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f057700a700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563929f6d000 0x563929e97b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f057700a700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563929f6d000 0x563929e97b80 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f0576008700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563929f6cc00 0x563929e97600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x56392a984030 tx=0x563929f23f80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f0578050f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f0578050f40 20 realm 2024-09-13T23:43:50.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f0578050f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.843+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d000 2024-09-13T23:43:50.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:50.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6dc00 con 0x563929f6d000 2024-09-13T23:43:50.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x563929f6d400 0x563929e98100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d400 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0576809700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x563929f6d400 0x563929e98100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0576809700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x563929f6d400 0x563929e98100 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6dc00 con 0x563929f6d400 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:50.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.847+0000 7f0578050f40 20 searching for the correct realm 2024-09-13T23:43:50.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.851+0000 7f0578050f40 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56392a024000 0x563929e98680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.851+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024400 con 0x56392a024000 2024-09-13T23:43:50.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.851+0000 7f0576008700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56392a024000 0x563929e98680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.851+0000 7f0576008700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56392a024000 0x563929e98680 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.851+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.851+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024400 con 0x563929f6d400 2024-09-13T23:43:50.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.851+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.851+0000 7f0576809700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56392a024400 0x563929e98c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.851+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024400 2024-09-13T23:43:50.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.851+0000 7f057700a700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56392a024400 0x563929e98c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.862 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.851+0000 7f057700a700 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56392a024400 0x563929e98c00 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.855+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:50.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.855+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d000 2024-09-13T23:43:50.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.855+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.855+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d000 2024-09-13T23:43:50.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.855+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.855+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024000 2024-09-13T23:43:50.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.855+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.855+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d000 2024-09-13T23:43:50.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.855+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.855+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d400 2024-09-13T23:43:50.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.855+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.855+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d400 2024-09-13T23:43:50.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d000 2024-09-13T23:43:50.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024400 2024-09-13T23:43:50.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:50.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d000 2024-09-13T23:43:50.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024400 2024-09-13T23:43:50.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:50.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d400 2024-09-13T23:43:50.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024000 2024-09-13T23:43:50.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.859+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d000 2024-09-13T23:43:50.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.863+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.863+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d400 2024-09-13T23:43:50.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.863+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.863+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024000 2024-09-13T23:43:50.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.863+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.863+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d000 2024-09-13T23:43:50.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.863+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.863+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024400 2024-09-13T23:43:50.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.863+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:50.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.863+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d000 2024-09-13T23:43:50.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.863+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.863+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024000 2024-09-13T23:43:50.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.867+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.867+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024400 2024-09-13T23:43:50.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.867+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:50.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.867+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d000 2024-09-13T23:43:50.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.867+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.867+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024400 2024-09-13T23:43:50.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.867+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:50.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.867+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024000 2024-09-13T23:43:50.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.867+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.867+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024400 2024-09-13T23:43:50.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.867+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:50.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.867+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d400 2024-09-13T23:43:50.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d400 2024-09-13T23:43:50.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024000 2024-09-13T23:43:50.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d400 2024-09-13T23:43:50.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x563929f6d400 2024-09-13T23:43:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0578050f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0578050f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0578050f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0578050f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0578050f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.871+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024800 con 0x563929f6d000 2024-09-13T23:43:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x563929f6d400 2024-09-13T23:43:50.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:50.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025000 con 0x563929f6d400 2024-09-13T23:43:50.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:50.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 20 zone default found 2024-09-13T23:43:50.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 4 Realm: () 2024-09-13T23:43:50.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:50.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:50.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:50.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 20 zonegroup default 2024-09-13T23:43:50.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x56392a024400 2024-09-13T23:43:50.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:50.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x56392a024000 2024-09-13T23:43:50.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:50.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.875+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025c00 con 0x56392a024000 2024-09-13T23:43:50.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.879+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.879+0000 7f0578050f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:50.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.879+0000 7f0578050f40 20 started sync module instance, tier type = 2024-09-13T23:43:50.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.879+0000 7f0578050f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:50.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.879+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x563929f6d400 2024-09-13T23:43:50.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.879+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.879+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94803516250880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025c00 con 0x563929f6d400 2024-09-13T23:43:50.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.879+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x563929f6d000 2024-09-13T23:43:50.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.883+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.883+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94803516252032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025000 con 0x563929f6d000 2024-09-13T23:43:50.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.883+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x563929f6d400 2024-09-13T23:43:50.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.883+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94803516250880] v35'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.887+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94803516252032] v35'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.887+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.887+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94803516253184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6dc00 con 0x563929f6d400 2024-09-13T23:43:50.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.887+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6d800 con 0x56392a024400 2024-09-13T23:43:50.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.891+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:50.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.891+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94803516254336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024800 con 0x56392a024400 2024-09-13T23:43:50.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.891+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x563929f6d400 2024-09-13T23:43:50.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.891+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94803516253184] v35'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.891+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94803516254336] v35'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:50.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.891+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.891+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94803516255488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x563929f6d400 2024-09-13T23:43:50.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.891+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025000 con 0x56392a024000 2024-09-13T23:43:50.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.895+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.895+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94803516256640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a026000 con 0x56392a024000 2024-09-13T23:43:50.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.895+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a026400 con 0x56392a024000 2024-09-13T23:43:50.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.895+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94803516255488] v35'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:50.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.895+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94803516256640] v35'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.895+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94803516257792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a026800 con 0x56392a024000 2024-09-13T23:43:50.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a026c00 con 0x56392a024400 2024-09-13T23:43:50.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94803516257792] v35'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:50.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'162 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:50.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94803516258944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027000 con 0x56392a024400 2024-09-13T23:43:50.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f0578050f40 20 add_watcher() i=0 2024-09-13T23:43:50.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f0578050f40 20 add_watcher() i=1 2024-09-13T23:43:50.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f0578050f40 20 add_watcher() i=2 2024-09-13T23:43:50.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f0578050f40 20 add_watcher() i=3 2024-09-13T23:43:50.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f0578050f40 20 add_watcher() i=4 2024-09-13T23:43:50.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f0578050f40 20 add_watcher() i=5 2024-09-13T23:43:50.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f0578050f40 20 add_watcher() i=6 2024-09-13T23:43:50.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.899+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94803516258944] v35'163 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:50.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f0578050f40 20 add_watcher() i=7 2024-09-13T23:43:50.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f0578050f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:50.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x563929f6d000 2024-09-13T23:43:50.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94803517947904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027800 con 0x563929f6d000 2024-09-13T23:43:50.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94803517947904] v35'157 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:50.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f055d7d7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:50.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f0578050f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:50.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f0578050f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:50.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f0578050f40 1 Processor -- start 2024-09-13T23:43:50.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f0578050f40 1 -- start start 2024-09-13T23:43:50.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f0578050f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 0x563929e99180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.903+0000 7f0578050f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563929c3e3c0 con 0x56392a026c00 2024-09-13T23:43:50.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0576008700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 0x563929e99180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0576008700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 0x563929e99180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52176/0 (socket says 172.21.15.167:52176) 2024-09-13T23:43:50.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0576008700 1 -- 172.21.15.167:0/3446595049 learned_addr learned my addr 172.21.15.167:0/3446595049 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:50.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0576008700 1 -- 172.21.15.167:0/3446595049 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563929bc2b60 con 0x56392a026c00 2024-09-13T23:43:50.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0576008700 1 --2- 172.21.15.167:0/3446595049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 0x563929e99180 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x56392a0360c0 tx=0x56392a038000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c7112ed557c397 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f055cfd6700 1 -- 172.21.15.167:0/3446595049 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56392a040000 con 0x56392a026c00 2024-09-13T23:43:50.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f055cfd6700 1 -- 172.21.15.167:0/3446595049 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563929f20600 con 0x56392a026c00 2024-09-13T23:43:50.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f055cfd6700 1 -- 172.21.15.167:0/3446595049 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56392a040160 con 0x56392a026c00 2024-09-13T23:43:50.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0578050f40 1 -- 172.21.15.167:0/3446595049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 msgr2=0x563929e99180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:50.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0578050f40 1 --2- 172.21.15.167:0/3446595049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 0x563929e99180 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x56392a0360c0 tx=0x56392a038000 comp rx=0 tx=0).stop 2024-09-13T23:43:50.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0578050f40 1 -- 172.21.15.167:0/3446595049 shutdown_connections 2024-09-13T23:43:50.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0578050f40 1 --2- 172.21.15.167:0/3446595049 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 0x563929e99180 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:50.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0578050f40 1 -- 172.21.15.167:0/3446595049 >> 172.21.15.167:0/3446595049 conn(0x56392a027c00 msgr2=0x563929cb2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:50.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0578050f40 1 -- 172.21.15.167:0/3446595049 shutdown_connections 2024-09-13T23:43:50.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0578050f40 1 -- 172.21.15.167:0/3446595049 wait complete. 2024-09-13T23:43:50.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0578050f40 1 Processor -- start 2024-09-13T23:43:50.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0578050f40 1 -- start start 2024-09-13T23:43:50.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0578050f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 0x563929e99180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.907+0000 7f0578050f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563929c3e500 con 0x56392a026c00 2024-09-13T23:43:50.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f0576008700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 0x563929e99180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f0576008700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 0x563929e99180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52190/0 (socket says 172.21.15.167:52190) 2024-09-13T23:43:50.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 learned_addr learned my addr 172.21.15.167:0/2595524699 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:50.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563929bc2d00 con 0x56392a026c00 2024-09-13T23:43:50.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f0576008700 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 0x563929e99180 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x56392a037380 tx=0x56392a0382a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f055b7d3700 1 -- 172.21.15.167:0/2595524699 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56392a0402c0 con 0x56392a026c00 2024-09-13T23:43:50.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f055b7d3700 1 -- 172.21.15.167:0/2595524699 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563929f20300 con 0x56392a026c00 2024-09-13T23:43:50.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563929bc2340 con 0x56392a026c00 2024-09-13T23:43:50.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f055b7d3700 1 -- 172.21.15.167:0/2595524699 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56392a040000 con 0x56392a026c00 2024-09-13T23:43:50.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563929bc2b60 con 0x56392a026c00 2024-09-13T23:43:50.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f055b7d3700 1 -- 172.21.15.167:0/2595524699 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x563929cabb80 con 0x56392a026c00 2024-09-13T23:43:50.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f055b7d3700 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56392a0ba000 0x56392a0c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f055b7d3700 1 -- 172.21.15.167:0/2595524699 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x563929c17500 con 0x56392a026c00 2024-09-13T23:43:50.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f057700a700 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56392a0ba000 0x56392a0c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f0578050f40 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56392a027000 0x563929e99700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x56392a027000 2024-09-13T23:43:50.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.911+0000 7f0576809700 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56392a027000 0x563929e99700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.915+0000 7f057700a700 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56392a0ba000 0x56392a0c0000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x563929faf7a0 tx=0x563929f233e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.915+0000 7f0576809700 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56392a027000 0x563929e99700 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.915+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v35'31 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.915+0000 7f0578050f40 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56392a026000 0x56392a0c0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.915+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a026400 con 0x56392a026000 2024-09-13T23:43:50.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.915+0000 7f0576008700 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56392a026000 0x56392a0c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.915+0000 7f0576008700 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56392a026000 0x56392a0c0580 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.919+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v35'19 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:50.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.919+0000 7f0578050f40 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56392a024800 0x56392a0c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.919+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025000 con 0x56392a024800 2024-09-13T23:43:50.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.919+0000 7f057700a700 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56392a024800 0x56392a0c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.919+0000 7f057700a700 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56392a024800 0x56392a0c0b00 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.923+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v35'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:50.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.923+0000 7f0578050f40 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563929f6d800 0x56392a0c1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:50.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.923+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x563929f6d800 2024-09-13T23:43:50.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.923+0000 7f0576809700 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563929f6d800 0x56392a0c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:50.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.923+0000 7f0576809700 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563929f6d800 0x56392a0c1080 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:50.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.927+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v35'158 uv15 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:50.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.927+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x56392a027000 2024-09-13T23:43:50.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.927+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v35'47 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.927+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x56392a024800 2024-09-13T23:43:50.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.931+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v35'34 uv10 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:50.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.931+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x56392a027000 2024-09-13T23:43:50.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.935+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v35'21 uv6 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.935+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb400 con 0x563929f6d800 2024-09-13T23:43:50.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.935+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v35'32 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:50.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.935+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb800 con 0x56392a027000 2024-09-13T23:43:50.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.939+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v35'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.939+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bbc00 con 0x56392a027000 2024-09-13T23:43:50.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.943+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v35'26 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.943+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x56392a027000 2024-09-13T23:43:50.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.943+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v35'48 uv13 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.943+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025c00 con 0x56392a024800 2024-09-13T23:43:50.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.947+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v35'22 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:50.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.947+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x56392a027000 2024-09-13T23:43:50.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.947+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v35'19 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.947+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x563929f6d800 2024-09-13T23:43:50.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.951+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v35'37 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:50.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.951+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x563929f6d800 2024-09-13T23:43:50.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.955+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v35'159 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:50.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.955+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6dc00 con 0x56392a027000 2024-09-13T23:43:50.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.955+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v35'41 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.955+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x56392a027000 2024-09-13T23:43:50.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.959+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v35'49 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.959+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x56392a024800 2024-09-13T23:43:50.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.963+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v35'20 uv18 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:50.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.963+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x56392a027000 2024-09-13T23:43:50.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.967+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v35'27 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.967+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb400 con 0x563929f6d800 2024-09-13T23:43:50.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.967+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v35'160 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:50.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.967+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bbc00 con 0x56392a027000 2024-09-13T23:43:50.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.971+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v35'42 uv9 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.971+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb800 con 0x56392a027000 2024-09-13T23:43:50.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.975+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v35'29 uv7 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.975+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x56392a024800 2024-09-13T23:43:50.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.979+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v35'21 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:50.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.979+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x56392a027000 2024-09-13T23:43:50.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.979+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v35'25 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:50.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.979+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x56392a024800 2024-09-13T23:43:50.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.983+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v35'15 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:50.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.983+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x56392a024800 2024-09-13T23:43:50.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.983+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v35'35 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:50.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.983+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x56392a024800 2024-09-13T23:43:50.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.987+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v35'21 uv3 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:50.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.987+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x56392a027000 2024-09-13T23:43:51.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.991+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v35'19 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.991+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb400 con 0x563929f6d800 2024-09-13T23:43:51.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.995+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v35'33 uv4 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.995+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bbc00 con 0x56392a027000 2024-09-13T23:43:51.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.995+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v35'43 uv11 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.995+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb800 con 0x563929f6d800 2024-09-13T23:43:51.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.999+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v35'38 uv8 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:50.999+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6dc00 con 0x56392a026000 2024-09-13T23:43:51.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.003+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v35'25 uv5 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.003+0000 7f0558fce700 20 reqs_thread_entry: start 2024-09-13T23:43:51.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.003+0000 7f0578050f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:51.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.007+0000 7f0578050f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:51.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.007+0000 7f0556fca700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:21 2024 2024-09-13T23:43:51.015 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:51.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.007+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=1 2024-09-13T23:43:51.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.007+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:33 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x56392a024800 2024-09-13T23:43:51.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.007+0000 7f05567a8700 20 reqs_thread_entry: start 2024-09-13T23:43:51.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.007+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(33 gc.5 [call] v35'36 uv36 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.007+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:34 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x56392a024800 2024-09-13T23:43:51.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.011+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(34 gc.5 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.011+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.011+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x56392a024800 2024-09-13T23:43:51.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.011+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(35 gc.5 [call out=24b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.011+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:36 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027800 con 0x56392a024800 2024-09-13T23:43:51.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.011+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(36 gc.5 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.011+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.011+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:37 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027800 con 0x56392a024800 2024-09-13T23:43:51.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.011+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(37 gc.5 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.011+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.011+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:38 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x56392a024800 2024-09-13T23:43:51.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.015+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(38 gc.5 [call] v35'37 uv37 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.015+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=1 2024-09-13T23:43:51.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.015+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x56392a027000 2024-09-13T23:43:51.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.015+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(39 gc.6 [call] v35'22 uv22 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.015+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x56392a027000 2024-09-13T23:43:51.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.019+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(40 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.019+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.019+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:41 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x56392a027000 2024-09-13T23:43:51.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.019+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(41 gc.6 [call out=24b] v0'0 uv22 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.019+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:42 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x56392a027000 2024-09-13T23:43:51.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.019+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(42 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.019+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.019+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:43 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x56392a027000 2024-09-13T23:43:51.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.019+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(43 gc.6 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.019+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.019+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x56392a027000 2024-09-13T23:43:51.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.023+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(44 gc.6 [call] v35'23 uv23 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.023+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=1 2024-09-13T23:43:51.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.023+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:45 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb400 con 0x563929f6d800 2024-09-13T23:43:51.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.023+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(45 gc.7 [call] v35'34 uv34 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.027+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bbc00 con 0x563929f6d800 2024-09-13T23:43:51.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.027+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(46 gc.7 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.027+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.027+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb800 con 0x563929f6d800 2024-09-13T23:43:51.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.027+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(47 gc.7 [call out=24b] v0'0 uv34 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.027+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:48 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x563929f6d800 2024-09-13T23:43:51.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.027+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(48 gc.7 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.027+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.027+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:49 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x563929f6d800 2024-09-13T23:43:51.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.027+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(49 gc.7 [call out=15b] v0'0 uv34 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.027+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.027+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:50 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x563929f6d800 2024-09-13T23:43:51.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.031+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(50 gc.7 [call] v35'35 uv35 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.031+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=1 2024-09-13T23:43:51.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.031+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:51 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x56392a027000 2024-09-13T23:43:51.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.035+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(51 gc.8 [call] v35'20 uv20 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.035+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x56392a027000 2024-09-13T23:43:51.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.035+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(52 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.035+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.035+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x56392a027000 2024-09-13T23:43:51.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.035+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(53 gc.8 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.035+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:54 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x56392a027000 2024-09-13T23:43:51.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.039+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(54 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.039+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.039+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:55 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x56392a027000 2024-09-13T23:43:51.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.039+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(55 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.039+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.039+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:56 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x56392a027000 2024-09-13T23:43:51.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.043+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(56 gc.8 [call] v35'21 uv21 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.043+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=1 2024-09-13T23:43:51.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.043+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:57 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x56392a027000 2024-09-13T23:43:51.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.043+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(57 gc.9 [call] v35'27 uv27 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.043+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:58 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb400 con 0x56392a027000 2024-09-13T23:43:51.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.047+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(58 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.047+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.047+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:59 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bbc00 con 0x56392a027000 2024-09-13T23:43:51.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.047+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(59 gc.9 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.047+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:60 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb800 con 0x56392a027000 2024-09-13T23:43:51.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.047+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 31 ==== osd_op_reply(60 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.047+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.047+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:61 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x56392a027000 2024-09-13T23:43:51.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.047+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 32 ==== osd_op_reply(61 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.047+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.047+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:62 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x56392a027000 2024-09-13T23:43:51.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.051+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 33 ==== osd_op_reply(62 gc.9 [call] v35'28 uv28 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.051+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=1 2024-09-13T23:43:51.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.051+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170000 con 0x56392a027000 2024-09-13T23:43:51.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.055+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 34 ==== osd_op_reply(63 gc.10 [call] v35'50 uv50 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.055+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170400 con 0x56392a027000 2024-09-13T23:43:51.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.055+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 35 ==== osd_op_reply(64 gc.10 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.055+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.055+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170800 con 0x56392a027000 2024-09-13T23:43:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.055+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 36 ==== osd_op_reply(65 gc.10 [call out=24b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.055+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170c00 con 0x56392a027000 2024-09-13T23:43:51.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.059+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 37 ==== osd_op_reply(66 gc.10 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.059+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.059+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171000 con 0x56392a027000 2024-09-13T23:43:51.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.059+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 38 ==== osd_op_reply(67 gc.10 [call out=15b] v0'0 uv50 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.059+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.059+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171400 con 0x56392a027000 2024-09-13T23:43:51.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.063+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 39 ==== osd_op_reply(68 gc.10 [call] v35'51 uv51 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.063+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=1 2024-09-13T23:43:51.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.063+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171800 con 0x56392a024800 2024-09-13T23:43:51.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.063+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(69 gc.11 [call] v35'23 uv23 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.063+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171c00 con 0x56392a024800 2024-09-13T23:43:51.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.067+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(70 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.067+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.067+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170000 con 0x56392a024800 2024-09-13T23:43:51.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.067+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(71 gc.11 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.067+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x56392a024800 2024-09-13T23:43:51.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.067+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(72 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.067+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.067+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x56392a024800 2024-09-13T23:43:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.067+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(73 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.067+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.067+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:74 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb400 con 0x56392a024800 2024-09-13T23:43:51.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.071+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(74 gc.11 [call] v35'24 uv24 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.071+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=1 2024-09-13T23:43:51.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.071+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bbc00 con 0x56392a027000 2024-09-13T23:43:51.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.075+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 40 ==== osd_op_reply(75 gc.12 [call] v35'20 uv20 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.075+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:76 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb800 con 0x56392a027000 2024-09-13T23:43:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.075+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 41 ==== osd_op_reply(76 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.075+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.075+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:77 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x56392a027000 2024-09-13T23:43:51.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.075+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 42 ==== osd_op_reply(77 gc.12 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.075+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:78 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x56392a027000 2024-09-13T23:43:51.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.079+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 43 ==== osd_op_reply(78 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.079+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.079+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:79 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x56392a027000 2024-09-13T23:43:51.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.079+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 44 ==== osd_op_reply(79 gc.12 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.079+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.079+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x56392a027000 2024-09-13T23:43:51.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.079+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 45 ==== osd_op_reply(80 gc.12 [call] v35'21 uv21 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.079+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=1 2024-09-13T23:43:51.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.079+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:81 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x563929f6d800 2024-09-13T23:43:51.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(81 gc.13 [call] v35'39 uv39 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:82 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bbc00 con 0x563929f6d800 2024-09-13T23:43:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(82 gc.13 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:83 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb800 con 0x563929f6d800 2024-09-13T23:43:51.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(83 gc.13 [call out=24b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:84 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x563929f6d800 2024-09-13T23:43:51.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(84 gc.13 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:85 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x563929f6d800 2024-09-13T23:43:51.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(85 gc.13 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.083+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:86 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x563929f6d800 2024-09-13T23:43:51.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.087+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(86 gc.13 [call] v35'40 uv40 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.087+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=1 2024-09-13T23:43:51.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.087+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:87 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x563929f6d800 2024-09-13T23:43:51.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.087+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(87 gc.14 [call] v35'161 uv161 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.087+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:88 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb400 con 0x563929f6d800 2024-09-13T23:43:51.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.087+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(88 gc.14 [call out=15b] v0'0 uv161 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.087+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.087+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:89 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x563929f6d800 2024-09-13T23:43:51.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.091+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(89 gc.14 [call out=24b] v0'0 uv161 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.091+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:90 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x563929f6d800 2024-09-13T23:43:51.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.091+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(90 gc.14 [call out=15b] v0'0 uv161 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.091+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.091+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:91 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x563929f6d800 2024-09-13T23:43:51.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.091+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(91 gc.14 [call out=15b] v0'0 uv161 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.091+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.091+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:92 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x563929f6d800 2024-09-13T23:43:51.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.095+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(92 gc.14 [call] v35'162 uv162 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.095+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=1 2024-09-13T23:43:51.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.095+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:93 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027800 con 0x56392a027000 2024-09-13T23:43:51.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.095+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 46 ==== osd_op_reply(93 gc.15 [call] v35'44 uv44 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.095+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:94 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170000 con 0x56392a027000 2024-09-13T23:43:51.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.095+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 47 ==== osd_op_reply(94 gc.15 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.095+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.095+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:95 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170400 con 0x56392a027000 2024-09-13T23:43:51.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.099+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 48 ==== osd_op_reply(95 gc.15 [call out=24b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.099+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:96 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170800 con 0x56392a027000 2024-09-13T23:43:51.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.099+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 49 ==== osd_op_reply(96 gc.15 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.099+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.099+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:97 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170c00 con 0x56392a027000 2024-09-13T23:43:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.099+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 50 ==== osd_op_reply(97 gc.15 [call out=15b] v0'0 uv44 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.099+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.099+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:98 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171000 con 0x56392a027000 2024-09-13T23:43:51.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.103+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 51 ==== osd_op_reply(98 gc.15 [call] v35'45 uv45 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.103+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=1 2024-09-13T23:43:51.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.103+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:99 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171400 con 0x56392a027000 2024-09-13T23:43:51.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.103+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 52 ==== osd_op_reply(99 gc.16 [call] v35'52 uv52 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.103+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:100 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171800 con 0x56392a027000 2024-09-13T23:43:51.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.107+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 53 ==== osd_op_reply(100 gc.16 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.107+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.107+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:101 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171c00 con 0x56392a027000 2024-09-13T23:43:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.107+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 54 ==== osd_op_reply(101 gc.16 [call out=24b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.107+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:102 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170000 con 0x56392a027000 2024-09-13T23:43:51.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.107+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 55 ==== osd_op_reply(102 gc.16 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.107+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.107+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:103 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x56392a027000 2024-09-13T23:43:51.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.107+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 56 ==== osd_op_reply(103 gc.16 [call out=15b] v0'0 uv52 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.107+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.107+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:104 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027800 con 0x56392a027000 2024-09-13T23:43:51.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.111+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 57 ==== osd_op_reply(104 gc.16 [call] v35'53 uv53 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.111+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=1 2024-09-13T23:43:51.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.111+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:105 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x56392a024800 2024-09-13T23:43:51.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(105 gc.17 [call] v35'21 uv21 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:106 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x56392a024800 2024-09-13T23:43:51.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(106 gc.17 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:107 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x56392a024800 2024-09-13T23:43:51.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(107 gc.17 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:108 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bbc00 con 0x56392a024800 2024-09-13T23:43:51.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== osd_op_reply(108 gc.17 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.125 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:109 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb800 con 0x56392a024800 2024-09-13T23:43:51.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 25 ==== osd_op_reply(109 gc.17 [call out=1700b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+1700 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-09-13T23:43:51.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 20 garbage collection: RGWGC::process iterating over entry tag='9144208c-3c85-4fb6-9007-2693d5a930f0.4222.959130024862934607', time=2024-09-13T23:43:50.392564+0000, chain.objs.size()=7 2024-09-13T23:43:51.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_1 2024-09-13T23:43:51.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:110 3.3es0 3:7df94ff6:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_1:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x563929f6d800 2024-09-13T23:43:51.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_2 2024-09-13T23:43:51.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:111 3.3bs0 3:ddca0762:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_2:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x56392a027000 2024-09-13T23:43:51.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_3 2024-09-13T23:43:51.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:112 3.1s0 3:83c058a2:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_3:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x563929f6d800 2024-09-13T23:43:51.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_4 2024-09-13T23:43:51.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:113 3.34s0 3:2d40dc11:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_4:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x56392a027000 2024-09-13T23:43:51.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_5 2024-09-13T23:43:51.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:114 3.38s0 3:1c86e2de:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_5:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb400 con 0x56392a024800 2024-09-13T23:43:51.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_6 2024-09-13T23:43:51.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:115 3.1es0 3:7aadbe2d:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_6:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x56392a024800 2024-09-13T23:43:51.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_7 2024-09-13T23:43:51.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.115+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:116 3.1ds0 3:b8fc95c2:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_7:head [call refcount.put in=83b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x56392a027000 2024-09-13T23:43:51.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.123+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 58 ==== osd_op_reply(113 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_4 [call] v35'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.123+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 59 ==== osd_op_reply(116 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_7 [call] v35'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.127+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 26 ==== osd_op_reply(114 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_5 [call] v35'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.127+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(110 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_1 [call] v35'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.127+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(112 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_3 [call] v35'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.127+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 60 ==== osd_op_reply(111 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_2 [call] v35'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.127+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 27 ==== osd_op_reply(115 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.1__shadow_.gvv8ub7SxSHWkmM0vaqzfrAYn7iF3N9_6 [call] v35'2 uv2 ondisk = 0) v8 ==== 230+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.127+0000 7f0578050f40 5 garbage collection: RGWGC::process removing entries, marker: 2024-09-13T23:43:51.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.127+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:117 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x56392a024800 2024-09-13T23:43:51.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.135+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 28 ==== osd_op_reply(117 gc.17 [call] v35'22 uv22 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.135+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:118 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x56392a024800 2024-09-13T23:43:51.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.135+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 29 ==== osd_op_reply(118 gc.17 [call] v35'23 uv23 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.135+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=1 2024-09-13T23:43:51.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.135+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:119 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x56392a027000 2024-09-13T23:43:51.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.139+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 61 ==== osd_op_reply(119 gc.18 [call] v35'28 uv28 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.139+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:120 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027800 con 0x56392a027000 2024-09-13T23:43:51.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.139+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 62 ==== osd_op_reply(120 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.139+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.139+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:121 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bbc00 con 0x56392a027000 2024-09-13T23:43:51.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.139+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 63 ==== osd_op_reply(121 gc.18 [call out=24b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.139+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:122 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb800 con 0x56392a027000 2024-09-13T23:43:51.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.139+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 64 ==== osd_op_reply(122 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.139+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.139+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:123 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb400 con 0x56392a027000 2024-09-13T23:43:51.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.143+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 65 ==== osd_op_reply(123 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.143+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.143+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:124 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x56392a027000 2024-09-13T23:43:51.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.143+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 66 ==== osd_op_reply(124 gc.18 [call] v35'29 uv29 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.143+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=1 2024-09-13T23:43:51.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.143+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:125 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x563929f6d800 2024-09-13T23:43:51.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 28 ==== osd_op_reply(125 gc.19 [call] v35'163 uv163 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:126 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x563929f6d800 2024-09-13T23:43:51.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 29 ==== osd_op_reply(126 gc.19 [call out=15b] v0'0 uv163 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:127 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x563929f6d800 2024-09-13T23:43:51.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 30 ==== osd_op_reply(127 gc.19 [call out=24b] v0'0 uv163 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:128 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x563929f6d800 2024-09-13T23:43:51.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 31 ==== osd_op_reply(128 gc.19 [call out=15b] v0'0 uv163 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:129 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x563929f6d800 2024-09-13T23:43:51.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 32 ==== osd_op_reply(129 gc.19 [call out=15b] v0'0 uv163 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.147+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:130 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170000 con 0x563929f6d800 2024-09-13T23:43:51.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.151+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 33 ==== osd_op_reply(130 gc.19 [call] v35'164 uv164 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.151+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=1 2024-09-13T23:43:51.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.151+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:131 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170400 con 0x56392a027000 2024-09-13T23:43:51.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.151+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 67 ==== osd_op_reply(131 gc.20 [call] v35'46 uv46 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.151+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:132 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170800 con 0x56392a027000 2024-09-13T23:43:51.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.155+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 68 ==== osd_op_reply(132 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.155+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.155+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:133 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170c00 con 0x56392a027000 2024-09-13T23:43:51.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.155+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 69 ==== osd_op_reply(133 gc.20 [call out=24b] v0'0 uv46 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.155+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:134 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171000 con 0x56392a027000 2024-09-13T23:43:51.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.155+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 70 ==== osd_op_reply(134 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.155+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.155+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:135 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171400 con 0x56392a027000 2024-09-13T23:43:51.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.155+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 71 ==== osd_op_reply(135 gc.20 [call out=15b] v0'0 uv46 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.155+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.155+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:136 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171800 con 0x56392a027000 2024-09-13T23:43:51.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.159+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 72 ==== osd_op_reply(136 gc.20 [call] v35'47 uv47 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.159+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=1 2024-09-13T23:43:51.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.159+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:137 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171c00 con 0x56392a027000 2024-09-13T23:43:51.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 73 ==== osd_op_reply(137 gc.21 [call] v35'30 uv30 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:138 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170000 con 0x56392a027000 2024-09-13T23:43:51.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 74 ==== osd_op_reply(138 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:139 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170400 con 0x56392a027000 2024-09-13T23:43:51.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 75 ==== osd_op_reply(139 gc.21 [call out=24b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:140 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170800 con 0x56392a027000 2024-09-13T23:43:51.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 76 ==== osd_op_reply(140 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:141 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x56392a027000 2024-09-13T23:43:51.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 77 ==== osd_op_reply(141 gc.21 [call out=15b] v0'0 uv30 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.163+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:142 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x56392a027000 2024-09-13T23:43:51.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.167+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 78 ==== osd_op_reply(142 gc.21 [call] v35'31 uv31 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.167+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=1 2024-09-13T23:43:51.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.167+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:143 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x56392a024800 2024-09-13T23:43:51.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 30 ==== osd_op_reply(143 gc.22 [call] v35'22 uv22 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:144 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x56392a024800 2024-09-13T23:43:51.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 31 ==== osd_op_reply(144 gc.22 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:145 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bbc00 con 0x56392a024800 2024-09-13T23:43:51.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 32 ==== osd_op_reply(145 gc.22 [call out=24b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:146 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb800 con 0x56392a024800 2024-09-13T23:43:51.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 33 ==== osd_op_reply(146 gc.22 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:147 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb400 con 0x56392a024800 2024-09-13T23:43:51.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 34 ==== osd_op_reply(147 gc.22 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.171+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:148 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x56392a024800 2024-09-13T23:43:51.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.175+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 35 ==== osd_op_reply(148 gc.22 [call] v35'23 uv23 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.175+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=1 2024-09-13T23:43:51.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.175+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:149 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x56392a027000 2024-09-13T23:43:51.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.175+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 79 ==== osd_op_reply(149 gc.23 [call] v35'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.175+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:150 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x56392a027000 2024-09-13T23:43:51.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.179+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 80 ==== osd_op_reply(150 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.179+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.179+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:151 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x56392a027000 2024-09-13T23:43:51.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.179+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 81 ==== osd_op_reply(151 gc.23 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.179+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:152 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027800 con 0x56392a027000 2024-09-13T23:43:51.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.179+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 82 ==== osd_op_reply(152 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.179+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.179+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:153 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178000 con 0x56392a027000 2024-09-13T23:43:51.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.179+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 83 ==== osd_op_reply(153 gc.23 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.179+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.179+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:154 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178400 con 0x56392a027000 2024-09-13T23:43:51.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.183+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 84 ==== osd_op_reply(154 gc.23 [call] v35'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.183+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=1 2024-09-13T23:43:51.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.183+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:155 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178800 con 0x56392a024800 2024-09-13T23:43:51.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.187+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 36 ==== osd_op_reply(155 gc.24 [call] v35'16 uv16 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.187+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:156 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178c00 con 0x56392a024800 2024-09-13T23:43:51.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.187+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 37 ==== osd_op_reply(156 gc.24 [call out=15b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.187+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.187+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:157 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179000 con 0x56392a024800 2024-09-13T23:43:51.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.187+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 38 ==== osd_op_reply(157 gc.24 [call out=24b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.187+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:158 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179400 con 0x56392a024800 2024-09-13T23:43:51.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.187+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 39 ==== osd_op_reply(158 gc.24 [call out=15b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.187+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.187+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:159 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179800 con 0x56392a024800 2024-09-13T23:43:51.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.187+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 40 ==== osd_op_reply(159 gc.24 [call out=15b] v0'0 uv16 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.191+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.191+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:160 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179c00 con 0x56392a024800 2024-09-13T23:43:51.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.191+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 41 ==== osd_op_reply(160 gc.24 [call] v35'17 uv17 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.191+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=1 2024-09-13T23:43:51.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.191+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:161 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178800 con 0x56392a024800 2024-09-13T23:43:51.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.195+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 42 ==== osd_op_reply(161 gc.25 [call] v35'38 uv38 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.195+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:162 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178c00 con 0x56392a024800 2024-09-13T23:43:51.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.195+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 43 ==== osd_op_reply(162 gc.25 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.195+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.195+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:163 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179000 con 0x56392a024800 2024-09-13T23:43:51.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.195+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 44 ==== osd_op_reply(163 gc.25 [call out=24b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.195+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:164 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179400 con 0x56392a024800 2024-09-13T23:43:51.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.195+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 45 ==== osd_op_reply(164 gc.25 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.195+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.195+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:165 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178000 con 0x56392a024800 2024-09-13T23:43:51.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.199+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 46 ==== osd_op_reply(165 gc.25 [call out=15b] v0'0 uv38 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.199+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.199+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:166 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178400 con 0x56392a024800 2024-09-13T23:43:51.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.199+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 47 ==== osd_op_reply(166 gc.25 [call] v35'39 uv39 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.199+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=1 2024-09-13T23:43:51.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.199+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:167 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x56392a024800 2024-09-13T23:43:51.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.203+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 48 ==== osd_op_reply(167 gc.26 [call] v35'22 uv22 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.203+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:168 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027800 con 0x56392a024800 2024-09-13T23:43:51.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.203+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 49 ==== osd_op_reply(168 gc.26 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.203+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.203+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:169 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x56392a024800 2024-09-13T23:43:51.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.203+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 50 ==== osd_op_reply(169 gc.26 [call out=24b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.203+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:170 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bbc00 con 0x56392a024800 2024-09-13T23:43:51.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.203+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 51 ==== osd_op_reply(170 gc.26 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.203+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.203+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:171 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb800 con 0x56392a024800 2024-09-13T23:43:51.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.207+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 52 ==== osd_op_reply(171 gc.26 [call out=15b] v0'0 uv22 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.207+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.207+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb400 con 0x56392a024800 2024-09-13T23:43:51.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.207+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 53 ==== osd_op_reply(172 gc.26 [call] v35'23 uv23 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.207+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=1 2024-09-13T23:43:51.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.207+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:173 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x56392a027000 2024-09-13T23:43:51.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.211+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 85 ==== osd_op_reply(173 gc.27 [call] v35'20 uv20 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.211+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:174 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x56392a027000 2024-09-13T23:43:51.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.211+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 86 ==== osd_op_reply(174 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.211+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.211+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:175 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x56392a027000 2024-09-13T23:43:51.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.211+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 87 ==== osd_op_reply(175 gc.27 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.211+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:176 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x56392a027000 2024-09-13T23:43:51.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.215+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 88 ==== osd_op_reply(176 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.215+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.215+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x56392a027000 2024-09-13T23:43:51.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.215+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 89 ==== osd_op_reply(177 gc.27 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.215+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.215+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:178 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x56392a027000 2024-09-13T23:43:51.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.219+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 90 ==== osd_op_reply(178 gc.27 [call] v35'21 uv21 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.219+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=1 2024-09-13T23:43:51.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.219+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:179 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x563929f6d800 2024-09-13T23:43:51.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.219+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 34 ==== osd_op_reply(179 gc.28 [call] v35'36 uv36 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.219+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:180 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x563929f6d800 2024-09-13T23:43:51.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.219+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 35 ==== osd_op_reply(180 gc.28 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.219+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.219+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:181 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027800 con 0x563929f6d800 2024-09-13T23:43:51.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.219+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 36 ==== osd_op_reply(181 gc.28 [call out=24b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.219+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:182 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178000 con 0x563929f6d800 2024-09-13T23:43:51.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.223+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 37 ==== osd_op_reply(182 gc.28 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.223+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.223+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:183 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178400 con 0x563929f6d800 2024-09-13T23:43:51.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.223+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 38 ==== osd_op_reply(183 gc.28 [call out=15b] v0'0 uv36 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.223+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.223+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:184 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178800 con 0x563929f6d800 2024-09-13T23:43:51.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.223+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 39 ==== osd_op_reply(184 gc.28 [call] v35'37 uv37 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.223+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=1 2024-09-13T23:43:51.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.223+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:185 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178c00 con 0x56392a027000 2024-09-13T23:43:51.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.227+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 91 ==== osd_op_reply(185 gc.29 [call] v35'48 uv48 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.227+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:186 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179000 con 0x56392a027000 2024-09-13T23:43:51.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.227+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 92 ==== osd_op_reply(186 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.227+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.227+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:187 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179400 con 0x56392a027000 2024-09-13T23:43:51.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.227+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 93 ==== osd_op_reply(187 gc.29 [call out=24b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.227+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:188 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179800 con 0x56392a027000 2024-09-13T23:43:51.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.231+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 94 ==== osd_op_reply(188 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.231+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.231+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:189 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179c00 con 0x56392a027000 2024-09-13T23:43:51.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.231+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 95 ==== osd_op_reply(189 gc.29 [call out=15b] v0'0 uv48 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.231+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.231+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:190 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027400 con 0x56392a027000 2024-09-13T23:43:51.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.235+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 96 ==== osd_op_reply(190 gc.29 [call] v35'49 uv49 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.235+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=1 2024-09-13T23:43:51.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.235+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:191 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a027800 con 0x563929f6d800 2024-09-13T23:43:51.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.235+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 40 ==== osd_op_reply(191 gc.30 [call] v35'41 uv41 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.235+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:192 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bac00 con 0x563929f6d800 2024-09-13T23:43:51.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.239+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 41 ==== osd_op_reply(192 gc.30 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.239+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.239+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:193 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba400 con 0x563929f6d800 2024-09-13T23:43:51.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.239+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 42 ==== osd_op_reply(193 gc.30 [call out=24b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.239+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:194 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0bb000 con 0x563929f6d800 2024-09-13T23:43:51.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.239+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 43 ==== osd_op_reply(194 gc.30 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.239+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.239+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:195 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a0ba800 con 0x563929f6d800 2024-09-13T23:43:51.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.239+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 44 ==== osd_op_reply(195 gc.30 [call out=15b] v0'0 uv41 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.239+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.239+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:196 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025800 con 0x563929f6d800 2024-09-13T23:43:51.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.243+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 45 ==== osd_op_reply(196 gc.30 [call] v35'42 uv42 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.243+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=1 2024-09-13T23:43:51.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.243+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:197 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x56392a026000 2024-09-13T23:43:51.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.243+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(197 gc.31 [call] v35'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.243+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:198 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x56392a026000 2024-09-13T23:43:51.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.247+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(198 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.247+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.247+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:199 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x56392a026000 2024-09-13T23:43:51.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.247+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(199 gc.31 [call out=24b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.247+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:200 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x56392a026000 2024-09-13T23:43:51.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.247+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(200 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.247+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.247+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:201 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6dc00 con 0x56392a026000 2024-09-13T23:43:51.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.247+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(201 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.247+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.247+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:202 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a026400 con 0x56392a026000 2024-09-13T23:43:51.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.251+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(202 gc.31 [call] v35'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.251+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=1 2024-09-13T23:43:51.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.251+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:203 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178000 con 0x56392a027000 2024-09-13T23:43:51.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.251+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 97 ==== osd_op_reply(203 gc.0 [call] v35'32 uv32 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.251+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:204 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178400 con 0x56392a027000 2024-09-13T23:43:51.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.255+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 98 ==== osd_op_reply(204 gc.0 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.255+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.255+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:205 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178800 con 0x56392a027000 2024-09-13T23:43:51.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.255+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 99 ==== osd_op_reply(205 gc.0 [call out=24b] v0'0 uv32 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.255+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:206 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178c00 con 0x56392a027000 2024-09-13T23:43:51.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.255+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 100 ==== osd_op_reply(206 gc.0 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.255+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.255+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:207 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179000 con 0x56392a027000 2024-09-13T23:43:51.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.255+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 101 ==== osd_op_reply(207 gc.0 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.255+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.255+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:208 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179400 con 0x56392a027000 2024-09-13T23:43:51.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.259+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 102 ==== osd_op_reply(208 gc.0 [call] v35'33 uv33 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.259+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=1 2024-09-13T23:43:51.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.259+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:209 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179800 con 0x56392a026000 2024-09-13T23:43:51.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.259+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(209 gc.1 [call] v35'20 uv20 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.259+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:210 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179c00 con 0x56392a026000 2024-09-13T23:43:51.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.263+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(210 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.263+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.263+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:211 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179800 con 0x56392a026000 2024-09-13T23:43:51.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.263+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(211 gc.1 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.263+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:212 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a026400 con 0x56392a026000 2024-09-13T23:43:51.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.263+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(212 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.263+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.263+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:213 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563929f6dc00 con 0x56392a026000 2024-09-13T23:43:51.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.263+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(213 gc.1 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.263+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.263+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:214 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a025400 con 0x56392a026000 2024-09-13T23:43:51.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.267+0000 7f0576008700 1 -- 172.21.15.167:0/2595524699 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(214 gc.1 [call] v35'21 uv21 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a026000 2024-09-13T23:43:51.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.267+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=1 2024-09-13T23:43:51.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.267+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:215 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a024c00 con 0x56392a024800 2024-09-13T23:43:51.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.267+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 54 ==== osd_op_reply(215 gc.2 [call] v35'20 uv20 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.267+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:216 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170000 con 0x56392a024800 2024-09-13T23:43:51.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.271+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 55 ==== osd_op_reply(216 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.271+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.271+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:217 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170400 con 0x56392a024800 2024-09-13T23:43:51.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.271+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 56 ==== osd_op_reply(217 gc.2 [call out=24b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.271+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:218 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170800 con 0x56392a024800 2024-09-13T23:43:51.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.271+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 57 ==== osd_op_reply(218 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.271+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.271+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:219 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a170c00 con 0x56392a024800 2024-09-13T23:43:51.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.271+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 58 ==== osd_op_reply(219 gc.2 [call out=15b] v0'0 uv20 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.271+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.271+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:220 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171000 con 0x56392a024800 2024-09-13T23:43:51.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.275+0000 7f057700a700 1 -- 172.21.15.167:0/2595524699 <== osd.3 v2:172.21.15.167:6816/3688363524 59 ==== osd_op_reply(220 gc.2 [call] v35'21 uv21 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024800 2024-09-13T23:43:51.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.275+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=1 2024-09-13T23:43:51.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.275+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:221 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171400 con 0x563929f6d800 2024-09-13T23:43:51.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 46 ==== osd_op_reply(221 gc.3 [call] v35'165 uv165 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:222 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171800 con 0x563929f6d800 2024-09-13T23:43:51.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 47 ==== osd_op_reply(222 gc.3 [call out=15b] v0'0 uv165 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:223 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171c00 con 0x563929f6d800 2024-09-13T23:43:51.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 48 ==== osd_op_reply(223 gc.3 [call out=24b] v0'0 uv165 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:224 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a17e000 con 0x563929f6d800 2024-09-13T23:43:51.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 49 ==== osd_op_reply(224 gc.3 [call out=15b] v0'0 uv165 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:225 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a17e400 con 0x563929f6d800 2024-09-13T23:43:51.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 50 ==== osd_op_reply(225 gc.3 [call out=15b] v0'0 uv165 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.279+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:226 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a17e800 con 0x563929f6d800 2024-09-13T23:43:51.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.283+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.1 v2:172.21.15.167:6800/3875661733 51 ==== osd_op_reply(226 gc.3 [call] v35'166 uv166 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d800 2024-09-13T23:43:51.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.283+0000 7f0578050f40 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=1 2024-09-13T23:43:51.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.283+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:227 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a17ec00 con 0x56392a027000 2024-09-13T23:43:51.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.287+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 103 ==== osd_op_reply(227 gc.4 [call] v35'54 uv54 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.287+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:228 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a17f000 con 0x56392a027000 2024-09-13T23:43:51.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.287+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 104 ==== osd_op_reply(228 gc.4 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.287+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.287+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:229 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a17f400 con 0x56392a027000 2024-09-13T23:43:51.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.287+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 105 ==== osd_op_reply(229 gc.4 [call out=24b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.287+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:230 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a17f800 con 0x56392a027000 2024-09-13T23:43:51.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.287+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 106 ==== osd_op_reply(230 gc.4 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.287+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-09-13T23:43:51.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.287+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:231 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a17fc00 con 0x56392a027000 2024-09-13T23:43:51.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.291+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 107 ==== osd_op_reply(231 gc.4 [call out=15b] v0'0 uv54 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.291+0000 7f0578050f40 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-09-13T23:43:51.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.291+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:232 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a17e000 con 0x56392a027000 2024-09-13T23:43:51.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.291+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 <== osd.0 v2:172.21.15.12:6800/2649399468 108 ==== osd_op_reply(232 gc.4 [call] v35'55 uv55 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x56392a00d200 con 0x56392a027000 2024-09-13T23:43:51.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.295+0000 7f055d7d7700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:51.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.295+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94803517947904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a17e400 con 0x563929f6d000 2024-09-13T23:43:51.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.295+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 data_loggenerations_metadata [watch unwatch cookie 94803517947904] v35'167 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:51.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.299+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94803516250880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171400 con 0x563929f6d400 2024-09-13T23:43:51.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.299+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(63 notify.0 [watch unwatch cookie 94803516250880] v35'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:51.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.299+0000 7f0578050f40 20 remove_watcher() i=0 2024-09-13T23:43:51.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.299+0000 7f0578050f40 2 removed watcher, disabling cache 2024-09-13T23:43:51.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.299+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94803516252032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171800 con 0x563929f6d000 2024-09-13T23:43:51.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.303+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 notify.1 [watch unwatch cookie 94803516252032] v35'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x563929f6d000 2024-09-13T23:43:51.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.303+0000 7f0578050f40 20 remove_watcher() i=1 2024-09-13T23:43:51.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.303+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94803516253184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a171c00 con 0x563929f6d400 2024-09-13T23:43:51.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.303+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 notify.2 [watch unwatch cookie 94803516253184] v35'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:51.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.303+0000 7f0578050f40 20 remove_watcher() i=2 2024-09-13T23:43:51.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.307+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94803516254336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178000 con 0x56392a024400 2024-09-13T23:43:51.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.307+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(66 notify.3 [watch unwatch cookie 94803516254336] v35'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:51.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.307+0000 7f0578050f40 20 remove_watcher() i=3 2024-09-13T23:43:51.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.307+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94803516255488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178400 con 0x563929f6d400 2024-09-13T23:43:51.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.311+0000 7f0576809700 1 -- 172.21.15.167:0/4192659245 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(67 notify.4 [watch unwatch cookie 94803516255488] v35'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d200 con 0x563929f6d400 2024-09-13T23:43:51.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.311+0000 7f0578050f40 20 remove_watcher() i=4 2024-09-13T23:43:51.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.311+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94803516256640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178800 con 0x56392a024000 2024-09-13T23:43:51.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.315+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(68 notify.5 [watch unwatch cookie 94803516256640] v35'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:51.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.315+0000 7f0578050f40 20 remove_watcher() i=5 2024-09-13T23:43:51.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.315+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94803516257792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a178c00 con 0x56392a024000 2024-09-13T23:43:51.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.315+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(69 notify.6 [watch unwatch cookie 94803516257792] v35'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00d440 con 0x56392a024000 2024-09-13T23:43:51.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.315+0000 7f0578050f40 20 remove_watcher() i=6 2024-09-13T23:43:51.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.315+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94803516258944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56392a179000 con 0x56392a024400 2024-09-13T23:43:51.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f057700a700 1 -- 172.21.15.167:0/4192659245 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(70 notify.7 [watch unwatch cookie 94803516258944] v35'164 uv162 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56392a00cb40 con 0x56392a024400 2024-09-13T23:43:51.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 20 remove_watcher() i=7 2024-09-13T23:43:51.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x563929f6d400 msgr2=0x563929e98100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x563929f6d400 0x563929e98100 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563929f6d000 msgr2=0x563929e97b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563929f6d000 0x563929e97b80 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56392a024400 msgr2=0x563929e98c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56392a024400 0x563929e98c00 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56392a024000 msgr2=0x563929e98680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56392a024000 0x563929e98680 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563929f6cc00 msgr2=0x563929e97600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563929f6cc00 0x563929e97600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x56392a984030 tx=0x563929f23f80 comp rx=0 tx=0).stop 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 msgr2=0x563929e96b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/4192659245 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1800 0x563929e96b00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x563929eccc00 tx=0x563929cb5b60 comp rx=0 tx=0).stop 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 reap_dead start 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0576008700 1 -- 172.21.15.167:0/4192659245 reap_dead start 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 shutdown_connections 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 >> 172.21.15.167:0/4192659245 conn(0x563929bb1400 msgr2=0x563929cb1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 shutdown_connections 2024-09-13T23:43:51.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/4192659245 wait complete. 2024-09-13T23:43:51.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56392a027000 msgr2=0x563929e99700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56392a027000 0x563929e99700 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563929f6d800 msgr2=0x56392a0c1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563929f6d800 0x56392a0c1080 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56392a026000 msgr2=0x56392a0c0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56392a026000 0x56392a0c0580 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56392a024800 msgr2=0x56392a0c0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56392a024800 0x56392a0c0b00 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56392a0ba000 msgr2=0x56392a0c0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56392a0ba000 0x56392a0c0000 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x563929faf7a0 tx=0x563929f233e0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0576809700 1 -- 172.21.15.167:0/2595524699 reap_dead start 2024-09-13T23:43:51.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 msgr2=0x563929e99180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 0x563929e99180 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x56392a037380 tx=0x56392a0382a0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 shutdown_connections 2024-09-13T23:43:51.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/2595524699 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56392a026c00 0x563929e99180 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 >> 172.21.15.167:0/2595524699 conn(0x56392a027c00 msgr2=0x563929cb2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 shutdown_connections 2024-09-13T23:43:51.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/2595524699 wait complete. 2024-09-13T23:43:51.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/1612550526 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563929f6c000 msgr2=0x563929e97080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/1612550526 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563929f6c000 0x563929e97080 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x563929ecd680 tx=0x563929f22120 comp rx=0 tx=0).stop 2024-09-13T23:43:51.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/1612550526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 msgr2=0x563929e96580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/1612550526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x56392a985ef0 tx=0x563929cb5920 comp rx=0 tx=0).stop 2024-09-13T23:43:51.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 -- 172.21.15.167:0/1612550526 shutdown_connections 2024-09-13T23:43:51.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/1612550526 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563929f6c000 0x563929e97080 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.319+0000 7f0578050f40 1 --2- 172.21.15.167:0/1612550526 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563929bb1000 0x563929e96580 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.323+0000 7f0578050f40 1 -- 172.21.15.167:0/1612550526 >> 172.21.15.167:0/1612550526 conn(0x563929bb0c00 msgr2=0x563929cb1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.323+0000 7f0578050f40 1 -- 172.21.15.167:0/1612550526 shutdown_connections 2024-09-13T23:43:51.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.323+0000 7f0578050f40 1 -- 172.21.15.167:0/1612550526 wait complete. 2024-09-13T23:43:51.335 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-09-13T23:43:51.336 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-09-13T23:43:51.336 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-09-13T23:43:51.373 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:51.373 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.363+0000 7f16558bbf40 1 Processor -- start 2024-09-13T23:43:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.363+0000 7f16558bbf40 1 -- start start 2024-09-13T23:43:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.363+0000 7f16558bbf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55bf62d8b400 0x55bf63070580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.363+0000 7f16558bbf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bf62e4f500 con 0x55bf62d8b000 2024-09-13T23:43:51.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.363+0000 7f16558bbf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bf62e18a00 con 0x55bf62d8b400 2024-09-13T23:43:51.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f1653873700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55bf62d8b400 0x55bf63070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f1653873700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55bf62d8b400 0x55bf63070580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52192/0 (socket says 172.21.15.167:52192) 2024-09-13T23:43:51.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f1653873700 1 -- 172.21.15.167:0/2356615794 learned_addr learned my addr 172.21.15.167:0/2356615794 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f1653873700 1 -- 172.21.15.167:0/2356615794 >> v1:172.21.15.12:6789/0 conn(0x55bf62d8b000 legacy=0x55bf62e8a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f1653072700 1 -- 172.21.15.167:0/2356615794 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155613633 0 0) 0x55bf62ee6000 con 0x55bf62d8b000 2024-09-13T23:43:51.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f1653873700 1 -- 172.21.15.167:0/2356615794 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf62d9dd40 con 0x55bf62d8b400 2024-09-13T23:43:51.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f1653873700 1 --2- 172.21.15.167:0/2356615794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b400 0x55bf63070580 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55bf6306b920 tx=0x55bf62e8f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86de051118c77fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f1653072700 1 -- 172.21.15.167:0/2356615794 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf62e2f4a0 con 0x55bf62d8b400 2024-09-13T23:43:51.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f1653072700 1 -- 172.21.15.167:0/2356615794 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf62e4f800 con 0x55bf62d8b400 2024-09-13T23:43:51.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f1653072700 1 -- 172.21.15.167:0/2356615794 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf62e2f600 con 0x55bf62d8b400 2024-09-13T23:43:51.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f16558bbf40 1 -- 172.21.15.167:0/2356615794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b400 msgr2=0x55bf63070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f16558bbf40 1 --2- 172.21.15.167:0/2356615794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b400 0x55bf63070580 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55bf6306b920 tx=0x55bf62e8f920 comp rx=0 tx=0).stop 2024-09-13T23:43:51.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f16558bbf40 1 -- 172.21.15.167:0/2356615794 shutdown_connections 2024-09-13T23:43:51.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f16558bbf40 1 --2- 172.21.15.167:0/2356615794 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b400 0x55bf63070580 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f16558bbf40 1 -- 172.21.15.167:0/2356615794 >> 172.21.15.167:0/2356615794 conn(0x55bf62d8ac00 msgr2=0x55bf62e8a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f16558bbf40 1 -- 172.21.15.167:0/2356615794 shutdown_connections 2024-09-13T23:43:51.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f16558bbf40 1 -- 172.21.15.167:0/2356615794 wait complete. 2024-09-13T23:43:51.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f16558bbf40 1 Processor -- start 2024-09-13T23:43:51.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f16558bbf40 1 -- start start 2024-09-13T23:43:51.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f16558bbf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 0x55bf63070580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.367+0000 7f16558bbf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf62e18500 con 0x55bf62d8b000 2024-09-13T23:43:51.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f1654074700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 0x55bf63070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f1654074700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 0x55bf63070580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52194/0 (socket says 172.21.15.167:52194) 2024-09-13T23:43:51.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f1654074700 1 -- 172.21.15.167:0/3874196672 learned_addr learned my addr 172.21.15.167:0/3874196672 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f1654074700 1 -- 172.21.15.167:0/3874196672 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf62d9dba0 con 0x55bf62d8b000 2024-09-13T23:43:51.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f1654074700 1 --2- 172.21.15.167:0/3874196672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 0x55bf63070580 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55bf63bebec0 tx=0x55bf62e8fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b6606a6e1169a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f1652871700 1 -- 172.21.15.167:0/3874196672 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf62e2f4a0 con 0x55bf62d8b000 2024-09-13T23:43:51.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f1652871700 1 -- 172.21.15.167:0/3874196672 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf62e4fb00 con 0x55bf62d8b000 2024-09-13T23:43:51.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f1652871700 1 -- 172.21.15.167:0/3874196672 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf62e2f600 con 0x55bf62d8b000 2024-09-13T23:43:51.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f16558bbf40 1 -- 172.21.15.167:0/3874196672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 msgr2=0x55bf63070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f16558bbf40 1 --2- 172.21.15.167:0/3874196672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 0x55bf63070580 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55bf63bebec0 tx=0x55bf62e8fb60 comp rx=0 tx=0).stop 2024-09-13T23:43:51.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f16558bbf40 1 -- 172.21.15.167:0/3874196672 shutdown_connections 2024-09-13T23:43:51.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f16558bbf40 1 --2- 172.21.15.167:0/3874196672 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 0x55bf63070580 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f16558bbf40 1 -- 172.21.15.167:0/3874196672 >> 172.21.15.167:0/3874196672 conn(0x55bf62d8ac00 msgr2=0x55bf62e8b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f16558bbf40 1 -- 172.21.15.167:0/3874196672 shutdown_connections 2024-09-13T23:43:51.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f16558bbf40 1 -- 172.21.15.167:0/3874196672 wait complete. 2024-09-13T23:43:51.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f16558bbf40 1 Processor -- start 2024-09-13T23:43:51.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f16558bbf40 1 -- start start 2024-09-13T23:43:51.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f16558bbf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 0x55bf63070580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f16558bbf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf62e183c0 con 0x55bf62d8b000 2024-09-13T23:43:51.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f1654074700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 0x55bf63070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f1654074700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 0x55bf63070580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52206/0 (socket says 172.21.15.167:52206) 2024-09-13T23:43:51.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.371+0000 7f1654074700 1 -- 172.21.15.167:0/866486522 learned_addr learned my addr 172.21.15.167:0/866486522 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f1654074700 1 -- 172.21.15.167:0/866486522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf62d9da00 con 0x55bf62d8b000 2024-09-13T23:43:51.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f1654074700 1 --2- 172.21.15.167:0/866486522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 0x55bf63070580 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x55bf63bf40f0 tx=0x55bf62e8fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f165106e700 1 -- 172.21.15.167:0/866486522 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf62e2f8c0 con 0x55bf62d8b000 2024-09-13T23:43:51.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f165106e700 1 -- 172.21.15.167:0/866486522 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf62e4fc80 con 0x55bf62d8b000 2024-09-13T23:43:51.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f16558bbf40 1 -- 172.21.15.167:0/866486522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bf62d9d860 con 0x55bf62d8b000 2024-09-13T23:43:51.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f165106e700 1 -- 172.21.15.167:0/866486522 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf62e2f4a0 con 0x55bf62d8b000 2024-09-13T23:43:51.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f16558bbf40 1 -- 172.21.15.167:0/866486522 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf62d9d6c0 con 0x55bf62d8b000 2024-09-13T23:43:51.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f16558bbf40 1 Processor -- start 2024-09-13T23:43:51.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f16558bbf40 1 -- start start 2024-09-13T23:43:51.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f16558bbf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 0x55bf63070b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f16558bbf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf62e18000 con 0x55bf62d8b800 2024-09-13T23:43:51.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f165106e700 1 -- 172.21.15.167:0/866486522 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55bf62e84b00 con 0x55bf62d8b000 2024-09-13T23:43:51.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f1654875700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 0x55bf63070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f165106e700 1 --2- 172.21.15.167:0/866486522 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63134000 0x55bf63071080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f1654875700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 0x55bf63070b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52210/0 (socket says 172.21.15.167:52210) 2024-09-13T23:43:51.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f1654875700 1 -- 172.21.15.167:0/2691111442 learned_addr learned my addr 172.21.15.167:0/2691111442 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f165106e700 1 -- 172.21.15.167:0/866486522 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55bf62df1180 con 0x55bf62d8b000 2024-09-13T23:43:51.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f1654074700 1 --2- 172.21.15.167:0/866486522 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63134000 0x55bf63071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.375+0000 7f1654875700 1 -- 172.21.15.167:0/2691111442 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf62d9c000 con 0x55bf62d8b800 2024-09-13T23:43:51.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f1654074700 1 --2- 172.21.15.167:0/866486522 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63134000 0x55bf63071080 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55bf63bf5050 tx=0x55bf630ea240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f1654875700 1 --2- 172.21.15.167:0/2691111442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 0x55bf63070b00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x55bf6313a060 tx=0x55bf630eade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bab574aad34014f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f164e869700 1 -- 172.21.15.167:0/2691111442 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf62e2f8c0 con 0x55bf62d8b800 2024-09-13T23:43:51.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f164e869700 1 -- 172.21.15.167:0/2691111442 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf6313c000 con 0x55bf62d8b800 2024-09-13T23:43:51.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f164e869700 1 -- 172.21.15.167:0/2691111442 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf62e2f4a0 con 0x55bf62d8b800 2024-09-13T23:43:51.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f16558bbf40 1 -- 172.21.15.167:0/2691111442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 msgr2=0x55bf63070b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f16558bbf40 1 --2- 172.21.15.167:0/2691111442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 0x55bf63070b00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x55bf6313a060 tx=0x55bf630eade0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f16558bbf40 1 -- 172.21.15.167:0/2691111442 shutdown_connections 2024-09-13T23:43:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f16558bbf40 1 --2- 172.21.15.167:0/2691111442 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 0x55bf63070b00 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f16558bbf40 1 -- 172.21.15.167:0/2691111442 >> 172.21.15.167:0/2691111442 conn(0x55bf62d8b400 msgr2=0x55bf62e8b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f16558bbf40 1 -- 172.21.15.167:0/2691111442 shutdown_connections 2024-09-13T23:43:51.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f16558bbf40 1 -- 172.21.15.167:0/2691111442 wait complete. 2024-09-13T23:43:51.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f16558bbf40 1 Processor -- start 2024-09-13T23:43:51.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f16558bbf40 1 -- start start 2024-09-13T23:43:51.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f16558bbf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 0x55bf63070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f16558bbf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf62db0640 con 0x55bf62d8b800 2024-09-13T23:43:51.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f1654875700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 0x55bf63070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f1654875700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 0x55bf63070b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52216/0 (socket says 172.21.15.167:52216) 2024-09-13T23:43:51.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.379+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 learned_addr learned my addr 172.21.15.167:0/3519934559 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf62d9cea0 con 0x55bf62d8b800 2024-09-13T23:43:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f1654875700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 0x55bf63070b00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x55bf63bf49c0 tx=0x55bf630ea840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f164d066700 1 -- 172.21.15.167:0/3519934559 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf62e2fb80 con 0x55bf62d8b800 2024-09-13T23:43:51.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bf62d9c820 con 0x55bf62d8b800 2024-09-13T23:43:51.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f164d066700 1 -- 172.21.15.167:0/3519934559 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf6313c180 con 0x55bf62d8b800 2024-09-13T23:43:51.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f164d066700 1 -- 172.21.15.167:0/3519934559 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf62e2f8c0 con 0x55bf62d8b800 2024-09-13T23:43:51.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf62d9c9c0 con 0x55bf62d8b800 2024-09-13T23:43:51.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f16558bbf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f164d066700 1 -- 172.21.15.167:0/3519934559 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55bf62e85080 con 0x55bf62d8b800 2024-09-13T23:43:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f164d066700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63134c00 0x55bf63071600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f164d066700 1 -- 172.21.15.167:0/3519934559 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55bf62df1340 con 0x55bf62d8b800 2024-09-13T23:43:51.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f16558bbf40 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bf63135000 0x55bf63071b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f1653873700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63134c00 0x55bf63071600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.383+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135400 con 0x55bf63135000 2024-09-13T23:43:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f1654875700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bf63135000 0x55bf63071b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f1654875700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bf63135000 0x55bf63071b80 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f1653873700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63134c00 0x55bf63071600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55bf6306b920 tx=0x55bf62e8f800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 20 realm 2024-09-13T23:43:51.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135000 2024-09-13T23:43:51.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:51.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135c00 con 0x55bf63135000 2024-09-13T23:43:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bf63135400 0x55bf63072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135400 2024-09-13T23:43:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f1654074700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bf63135400 0x55bf63072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f1654074700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bf63135400 0x55bf63072100 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:51.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.387+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135c00 con 0x55bf63135400 2024-09-13T23:43:51.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f16558bbf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:51.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f16558bbf40 20 searching for the correct realm 2024-09-13T23:43:51.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f16558bbf40 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bf630c8000 0x55bf63072680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c8400 con 0x55bf630c8000 2024-09-13T23:43:51.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f1653873700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bf630c8000 0x55bf63072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f1653873700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bf630c8000 0x55bf63072680 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c8400 con 0x55bf63135400 2024-09-13T23:43:51.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f1654074700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bf630c8400 0x55bf63072c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8400 2024-09-13T23:43:51.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f1654875700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bf630c8400 0x55bf63072c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f1654875700 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bf630c8400 0x55bf63072c00 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.391+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135000 2024-09-13T23:43:51.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.395+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.395+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135000 2024-09-13T23:43:51.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.395+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.395+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8000 2024-09-13T23:43:51.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.395+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.395+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135000 2024-09-13T23:43:51.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.395+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.395+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135400 2024-09-13T23:43:51.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.395+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.395+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135400 2024-09-13T23:43:51.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.395+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.395+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135000 2024-09-13T23:43:51.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8400 2024-09-13T23:43:51.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135000 2024-09-13T23:43:51.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8400 2024-09-13T23:43:51.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135400 2024-09-13T23:43:51.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8000 2024-09-13T23:43:51.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135000 2024-09-13T23:43:51.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.399+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135400 2024-09-13T23:43:51.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.403+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.403+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8000 2024-09-13T23:43:51.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.403+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.403+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135000 2024-09-13T23:43:51.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.403+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.403+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8400 2024-09-13T23:43:51.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.403+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.403+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135000 2024-09-13T23:43:51.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.403+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.403+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8000 2024-09-13T23:43:51.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.403+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.403+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8400 2024-09-13T23:43:51.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.407+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.407+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135000 2024-09-13T23:43:51.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.407+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.407+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8400 2024-09-13T23:43:51.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.407+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.407+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8000 2024-09-13T23:43:51.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.407+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.407+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8400 2024-09-13T23:43:51.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.407+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.407+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135400 2024-09-13T23:43:51.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.407+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.407+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135400 2024-09-13T23:43:51.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8000 2024-09-13T23:43:51.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135400 2024-09-13T23:43:51.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf63135400 2024-09-13T23:43:51.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f16558bbf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:51.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f16558bbf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:51.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f16558bbf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:51.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f16558bbf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:51.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f16558bbf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c8800 con 0x55bf63135000 2024-09-13T23:43:51.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f16558bbf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f16558bbf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c8c00 con 0x55bf63135400 2024-09-13T23:43:51.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f16558bbf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:51.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.411+0000 7f16558bbf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9000 con 0x55bf63135400 2024-09-13T23:43:51.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:51.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 20 zone default found 2024-09-13T23:43:51.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 4 Realm: () 2024-09-13T23:43:51.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:51.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:51.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:51.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 20 zonegroup default 2024-09-13T23:43:51.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9400 con 0x55bf630c8400 2024-09-13T23:43:51.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9800 con 0x55bf630c8000 2024-09-13T23:43:51.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9c00 con 0x55bf630c8000 2024-09-13T23:43:51.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 20 started sync module instance, tier type = 2024-09-13T23:43:51.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:51.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.415+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9800 con 0x55bf63135400 2024-09-13T23:43:51.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.419+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.419+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94280488483584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9c00 con 0x55bf63135400 2024-09-13T23:43:51.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.419+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c8c00 con 0x55bf63135000 2024-09-13T23:43:51.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.423+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.423+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94280488484736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9000 con 0x55bf63135000 2024-09-13T23:43:51.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.423+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9800 con 0x55bf63135400 2024-09-13T23:43:51.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.423+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94280488483584] v35'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.423+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94280488484736] v35'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.423+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.423+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94280488485888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135c00 con 0x55bf63135400 2024-09-13T23:43:51.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.423+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135800 con 0x55bf630c8400 2024-09-13T23:43:51.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.427+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.427+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94280488487040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c8800 con 0x55bf630c8400 2024-09-13T23:43:51.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.427+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9400 con 0x55bf63135400 2024-09-13T23:43:51.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.427+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94280488485888] v35'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.431+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94280488487040] v35'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.431+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.431+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94280488488192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c8c00 con 0x55bf63135400 2024-09-13T23:43:51.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.431+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9000 con 0x55bf630c8000 2024-09-13T23:43:51.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.431+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.431+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94280488489344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630ca000 con 0x55bf630c8000 2024-09-13T23:43:51.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.435+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630ca400 con 0x55bf630c8000 2024-09-13T23:43:51.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.435+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94280488488192] v35'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.435+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94280488489344] v35'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.435+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.435+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94280488490496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630ca800 con 0x55bf630c8000 2024-09-13T23:43:51.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.435+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cac00 con 0x55bf630c8400 2024-09-13T23:43:51.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94280488490496] v35'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'165 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94280488491648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb000 con 0x55bf630c8400 2024-09-13T23:43:51.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f16558bbf40 20 add_watcher() i=0 2024-09-13T23:43:51.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f16558bbf40 20 add_watcher() i=1 2024-09-13T23:43:51.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f16558bbf40 20 add_watcher() i=2 2024-09-13T23:43:51.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f16558bbf40 20 add_watcher() i=3 2024-09-13T23:43:51.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f16558bbf40 20 add_watcher() i=4 2024-09-13T23:43:51.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f16558bbf40 20 add_watcher() i=5 2024-09-13T23:43:51.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f16558bbf40 20 add_watcher() i=6 2024-09-13T23:43:51.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94280488491648] v35'166 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f16558bbf40 20 add_watcher() i=7 2024-09-13T23:43:51.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f16558bbf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:51.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb400 con 0x55bf63135000 2024-09-13T23:43:51.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.439+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94280490164224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb800 con 0x55bf63135000 2024-09-13T23:43:51.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.443+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94280490164224] v35'168 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.443+0000 7f163b042700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:51.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.443+0000 7f16558bbf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:51.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.443+0000 7f16558bbf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:51.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.443+0000 7f16558bbf40 1 Processor -- start 2024-09-13T23:43:51.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.443+0000 7f16558bbf40 1 -- start start 2024-09-13T23:43:51.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.443+0000 7f16558bbf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 0x55bf63073180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.443+0000 7f16558bbf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf62db0780 con 0x55bf630cac00 2024-09-13T23:43:51.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.443+0000 7f1653873700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 0x55bf63073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.443+0000 7f1653873700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 0x55bf63073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52228/0 (socket says 172.21.15.167:52228) 2024-09-13T23:43:51.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.443+0000 7f1653873700 1 -- 172.21.15.167:0/1007045375 learned_addr learned my addr 172.21.15.167:0/1007045375 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.443+0000 7f1653873700 1 -- 172.21.15.167:0/1007045375 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf62d9cb60 con 0x55bf630cac00 2024-09-13T23:43:51.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f1653873700 1 --2- 172.21.15.167:0/1007045375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 0x55bf63073180 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55bf6320a0c0 tx=0x55bf62e8f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ca37087a9ffb42 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f163a841700 1 -- 172.21.15.167:0/1007045375 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf62e2f760 con 0x55bf630cac00 2024-09-13T23:43:51.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f163a841700 1 -- 172.21.15.167:0/1007045375 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf6313c600 con 0x55bf630cac00 2024-09-13T23:43:51.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f163a841700 1 -- 172.21.15.167:0/1007045375 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf63212000 con 0x55bf630cac00 2024-09-13T23:43:51.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f16558bbf40 1 -- 172.21.15.167:0/1007045375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 msgr2=0x55bf63073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f16558bbf40 1 --2- 172.21.15.167:0/1007045375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 0x55bf63073180 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55bf6320a0c0 tx=0x55bf62e8f860 comp rx=0 tx=0).stop 2024-09-13T23:43:51.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f16558bbf40 1 -- 172.21.15.167:0/1007045375 shutdown_connections 2024-09-13T23:43:51.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f16558bbf40 1 --2- 172.21.15.167:0/1007045375 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 0x55bf63073180 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f16558bbf40 1 -- 172.21.15.167:0/1007045375 >> 172.21.15.167:0/1007045375 conn(0x55bf630cbc00 msgr2=0x55bf62e8c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f16558bbf40 1 -- 172.21.15.167:0/1007045375 shutdown_connections 2024-09-13T23:43:51.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f16558bbf40 1 -- 172.21.15.167:0/1007045375 wait complete. 2024-09-13T23:43:51.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f16558bbf40 1 Processor -- start 2024-09-13T23:43:51.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f16558bbf40 1 -- start start 2024-09-13T23:43:51.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f16558bbf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 0x55bf63073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f16558bbf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf62e18000 con 0x55bf630cac00 2024-09-13T23:43:51.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f1653873700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 0x55bf63073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f1653873700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 0x55bf63073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52246/0 (socket says 172.21.15.167:52246) 2024-09-13T23:43:51.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f1653873700 1 -- 172.21.15.167:0/995183761 learned_addr learned my addr 172.21.15.167:0/995183761 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f1653873700 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf62d9cd00 con 0x55bf630cac00 2024-09-13T23:43:51.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f1653873700 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 0x55bf63073180 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55bf6320a2d0 tx=0x55bf631dc420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f163903e700 1 -- 172.21.15.167:0/995183761 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf63212160 con 0x55bf630cac00 2024-09-13T23:43:51.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.447+0000 7f163903e700 1 -- 172.21.15.167:0/995183761 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bf6313c300 con 0x55bf630cac00 2024-09-13T23:43:51.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.451+0000 7f163903e700 1 -- 172.21.15.167:0/995183761 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bf632122c0 con 0x55bf630cac00 2024-09-13T23:43:51.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.451+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bf62d9c340 con 0x55bf630cac00 2024-09-13T23:43:51.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.451+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf62d9dd40 con 0x55bf630cac00 2024-09-13T23:43:51.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.451+0000 7f163903e700 1 -- 172.21.15.167:0/995183761 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55bf62e85b80 con 0x55bf630cac00 2024-09-13T23:43:51.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.451+0000 7f163903e700 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63288000 0x55bf6328e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.451+0000 7f163903e700 1 -- 172.21.15.167:0/995183761 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55bf62df1500 con 0x55bf630cac00 2024-09-13T23:43:51.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.451+0000 7f1654875700 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63288000 0x55bf6328e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.451+0000 7f16558bbf40 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bf630cb000 0x55bf63073700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.451+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb400 con 0x55bf630cb000 2024-09-13T23:43:51.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.451+0000 7f1654074700 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bf630cb000 0x55bf63073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.451+0000 7f1654875700 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63288000 0x55bf6328e000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55bf6313b6e0 tx=0x55bf630eb7a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.451+0000 7f1654074700 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bf630cb000 0x55bf63073700 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.455+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v35'34 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.455+0000 7f16558bbf40 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bf630ca000 0x55bf6328e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.455+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630ca400 con 0x55bf630ca000 2024-09-13T23:43:51.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.455+0000 7f1653873700 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bf630ca000 0x55bf6328e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.455+0000 7f1653873700 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bf630ca000 0x55bf6328e580 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.459+0000 7f1653873700 1 -- 172.21.15.167:0/995183761 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v35'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630ca000 2024-09-13T23:43:51.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.459+0000 7f16558bbf40 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bf630c8800 0x55bf6328eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.459+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9000 con 0x55bf630c8800 2024-09-13T23:43:51.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.459+0000 7f1654875700 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bf630c8800 0x55bf6328eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.459+0000 7f1654875700 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bf630c8800 0x55bf6328eb00 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.459+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v35'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.459+0000 7f16558bbf40 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bf63135800 0x55bf6328f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.459+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288400 con 0x55bf63135800 2024-09-13T23:43:51.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.459+0000 7f1654074700 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bf63135800 0x55bf6328f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.463+0000 7f1654074700 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bf63135800 0x55bf6328f080 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.463+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v35'169 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.463+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288800 con 0x55bf630cb000 2024-09-13T23:43:51.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.467+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v35'56 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.467+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288c00 con 0x55bf630c8800 2024-09-13T23:43:51.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.467+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v35'40 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.467+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289000 con 0x55bf630cb000 2024-09-13T23:43:51.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.471+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v35'24 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.471+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289400 con 0x55bf63135800 2024-09-13T23:43:51.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.475+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v35'38 uv35 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.475+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289800 con 0x55bf630cb000 2024-09-13T23:43:51.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.475+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v35'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.475+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289c00 con 0x55bf630cb000 2024-09-13T23:43:51.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.479+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v35'29 uv28 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.479+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb400 con 0x55bf630cb000 2024-09-13T23:43:51.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.483+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v35'57 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.483+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9c00 con 0x55bf630c8800 2024-09-13T23:43:51.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.483+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v35'25 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.483+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9800 con 0x55bf630cb000 2024-09-13T23:43:51.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.487+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v35'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.487+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9400 con 0x55bf63135800 2024-09-13T23:43:51.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.491+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v35'43 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.491+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c8c00 con 0x55bf63135800 2024-09-13T23:43:51.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.491+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v35'170 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.491+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135c00 con 0x55bf630cb000 2024-09-13T23:43:51.504 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.495+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v35'50 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.495+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb400 con 0x55bf630cb000 2024-09-13T23:43:51.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.499+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v35'58 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.499+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288400 con 0x55bf630c8800 2024-09-13T23:43:51.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.499+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v35'24 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.499+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288800 con 0x55bf630cb000 2024-09-13T23:43:51.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.503+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v35'30 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.503+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289000 con 0x55bf63135800 2024-09-13T23:43:51.515 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.507+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v35'171 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.507+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289400 con 0x55bf630cb000 2024-09-13T23:43:51.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.507+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v35'51 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.511+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289800 con 0x55bf630cb000 2024-09-13T23:43:51.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.511+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v35'32 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.511+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289c00 con 0x55bf630c8800 2024-09-13T23:43:51.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.515+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v35'24 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.524 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.515+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288c00 con 0x55bf630cb000 2024-09-13T23:43:51.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.519+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v35'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.527 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.519+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288400 con 0x55bf630c8800 2024-09-13T23:43:51.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.519+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v35'18 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.530 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.519+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289c00 con 0x55bf630c8800 2024-09-13T23:43:51.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.523+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v35'41 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.532 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.523+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288800 con 0x55bf630c8800 2024-09-13T23:43:51.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.527+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v35'24 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.535 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.527+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289000 con 0x55bf630cb000 2024-09-13T23:43:51.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.527+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v35'22 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.538 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.527+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289400 con 0x55bf63135800 2024-09-13T23:43:51.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.531+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v35'39 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.541 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.531+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb800 con 0x55bf630cb000 2024-09-13T23:43:51.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.535+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v35'52 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.535+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb400 con 0x55bf63135800 2024-09-13T23:43:51.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.539+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v35'44 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.539+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135c00 con 0x55bf630ca000 2024-09-13T23:43:51.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.539+0000 7f1653873700 1 -- 172.21.15.167:0/995183761 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v35'28 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630ca000 2024-09-13T23:43:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.543+0000 7f1636839700 20 reqs_thread_entry: start 2024-09-13T23:43:51.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.543+0000 7f16558bbf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:51.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.543+0000 7f16558bbf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:51.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.543+0000 7f1634835700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:21 2024 2024-09-13T23:43:51.553 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:51.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.543+0000 7f1634013700 20 reqs_thread_entry: start 2024-09-13T23:43:51.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.543+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9800 con 0x55bf630cb000 2024-09-13T23:43:51.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.543+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(33 gc.0 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.543+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9400 con 0x55bf630cb000 2024-09-13T23:43:51.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.547+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(34 gc.0 [call out=24b] v0'0 uv33 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.547+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:35 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c8c00 con 0x55bf630cb000 2024-09-13T23:43:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.547+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(35 gc.0 [call out=15b] v0'0 uv33 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.547+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:36 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9800 con 0x55bf630ca000 2024-09-13T23:43:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.547+0000 7f1653873700 1 -- 172.21.15.167:0/995183761 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(36 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630ca000 2024-09-13T23:43:51.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.547+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:37 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9400 con 0x55bf630ca000 2024-09-13T23:43:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.547+0000 7f1653873700 1 -- 172.21.15.167:0/995183761 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(37 gc.1 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630ca000 2024-09-13T23:43:51.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.547+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:38 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c8c00 con 0x55bf630ca000 2024-09-13T23:43:51.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.547+0000 7f1653873700 1 -- 172.21.15.167:0/995183761 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(38 gc.1 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630ca000 2024-09-13T23:43:51.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.547+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:39 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb800 con 0x55bf630c8800 2024-09-13T23:43:51.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.551+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(39 gc.2 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.551+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:40 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb400 con 0x55bf630c8800 2024-09-13T23:43:51.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.551+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(40 gc.2 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.551+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:41 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289800 con 0x55bf630c8800 2024-09-13T23:43:51.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.551+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(41 gc.2 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.551+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:42 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288c00 con 0x55bf63135800 2024-09-13T23:43:51.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.551+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(42 gc.3 [call out=15b] v0'0 uv166 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.551+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:43 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289000 con 0x55bf63135800 2024-09-13T23:43:51.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.551+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(43 gc.3 [call out=24b] v0'0 uv166 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.551+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:44 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289400 con 0x55bf63135800 2024-09-13T23:43:51.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.555+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(44 gc.3 [call out=15b] v0'0 uv166 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.555+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9800 con 0x55bf630cb000 2024-09-13T23:43:51.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.555+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(45 gc.4 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.555+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:46 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9400 con 0x55bf630cb000 2024-09-13T23:43:51.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.555+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(46 gc.4 [call out=24b] v0'0 uv55 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.555+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:47 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135c00 con 0x55bf630cb000 2024-09-13T23:43:51.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.555+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(47 gc.4 [call out=15b] v0'0 uv55 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.555+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:48 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630ca400 con 0x55bf630c8800 2024-09-13T23:43:51.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.555+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(48 gc.5 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.559+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:49 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb800 con 0x55bf630c8800 2024-09-13T23:43:51.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.559+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(49 gc.5 [call out=24b] v0'0 uv37 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.559+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:50 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb400 con 0x55bf630c8800 2024-09-13T23:43:51.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.559+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(50 gc.5 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.559+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:51 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630ca400 con 0x55bf630cb000 2024-09-13T23:43:51.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.559+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(51 gc.6 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.559+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288400 con 0x55bf630cb000 2024-09-13T23:43:51.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.559+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(52 gc.6 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.559+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289c00 con 0x55bf630cb000 2024-09-13T23:43:51.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.563+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(53 gc.6 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.563+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:54 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288800 con 0x55bf63135800 2024-09-13T23:43:51.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.563+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(54 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.563+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:55 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289800 con 0x55bf63135800 2024-09-13T23:43:51.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.563+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(55 gc.7 [call out=24b] v0'0 uv35 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.563+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:56 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288c00 con 0x55bf63135800 2024-09-13T23:43:51.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.563+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(56 gc.7 [call out=15b] v0'0 uv35 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.563+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:57 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289000 con 0x55bf630cb000 2024-09-13T23:43:51.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.563+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(57 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.563+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:58 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289400 con 0x55bf630cb000 2024-09-13T23:43:51.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.563+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(58 gc.8 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.567+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:59 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288400 con 0x55bf630cb000 2024-09-13T23:43:51.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.567+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(59 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.567+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:60 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289c00 con 0x55bf630cb000 2024-09-13T23:43:51.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.567+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(60 gc.9 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.567+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:61 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288800 con 0x55bf630cb000 2024-09-13T23:43:51.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.567+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(61 gc.9 [call out=24b] v0'0 uv28 ondisk = 0) v8 ==== 148+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.567+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:62 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630ca400 con 0x55bf630cb000 2024-09-13T23:43:51.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.567+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(62 gc.9 [call out=15b] v0'0 uv28 ondisk = 0) v8 ==== 148+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.567+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135c00 con 0x55bf630cb000 2024-09-13T23:43:51.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.571+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 31 ==== osd_op_reply(63 gc.10 [call out=15b] v0'0 uv51 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.571+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9800 con 0x55bf630cb000 2024-09-13T23:43:51.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.571+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 32 ==== osd_op_reply(64 gc.10 [call out=24b] v0'0 uv51 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.571+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9400 con 0x55bf630cb000 2024-09-13T23:43:51.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.571+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 33 ==== osd_op_reply(65 gc.10 [call out=15b] v0'0 uv51 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.571+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:66 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9800 con 0x55bf630c8800 2024-09-13T23:43:51.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.571+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(66 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.571+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9400 con 0x55bf630c8800 2024-09-13T23:43:51.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.575+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(67 gc.11 [call out=24b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.575+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135c00 con 0x55bf630c8800 2024-09-13T23:43:51.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.575+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(68 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.575+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630ca400 con 0x55bf630cb000 2024-09-13T23:43:51.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.575+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 34 ==== osd_op_reply(69 gc.12 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.575+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289800 con 0x55bf630cb000 2024-09-13T23:43:51.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.575+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 35 ==== osd_op_reply(70 gc.12 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.575+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288c00 con 0x55bf630cb000 2024-09-13T23:43:51.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.579+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 36 ==== osd_op_reply(71 gc.12 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.579+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289000 con 0x55bf63135800 2024-09-13T23:43:51.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.579+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(72 gc.13 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.579+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289400 con 0x55bf63135800 2024-09-13T23:43:51.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.579+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(73 gc.13 [call out=24b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.579+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:74 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288400 con 0x55bf63135800 2024-09-13T23:43:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.579+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(74 gc.13 [call out=15b] v0'0 uv40 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.579+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:75 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289c00 con 0x55bf63135800 2024-09-13T23:43:51.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.579+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(75 gc.14 [call out=15b] v0'0 uv162 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.579+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288800 con 0x55bf63135800 2024-09-13T23:43:51.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.583+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(76 gc.14 [call out=24b] v0'0 uv162 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.583+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:77 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334000 con 0x55bf63135800 2024-09-13T23:43:51.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.583+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(77 gc.14 [call out=15b] v0'0 uv162 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.583+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:78 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334400 con 0x55bf630cb000 2024-09-13T23:43:51.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.583+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 37 ==== osd_op_reply(78 gc.15 [call out=15b] v0'0 uv45 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.583+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:79 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334800 con 0x55bf630cb000 2024-09-13T23:43:51.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.583+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 38 ==== osd_op_reply(79 gc.15 [call out=24b] v0'0 uv45 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.583+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334c00 con 0x55bf630cb000 2024-09-13T23:43:51.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.583+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 39 ==== osd_op_reply(80 gc.15 [call out=15b] v0'0 uv45 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.583+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:81 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63335000 con 0x55bf630cb000 2024-09-13T23:43:51.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.587+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 40 ==== osd_op_reply(81 gc.16 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.587+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:82 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63335400 con 0x55bf630cb000 2024-09-13T23:43:51.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.587+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 41 ==== osd_op_reply(82 gc.16 [call out=24b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.587+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:83 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63335800 con 0x55bf630cb000 2024-09-13T23:43:51.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.587+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 42 ==== osd_op_reply(83 gc.16 [call out=15b] v0'0 uv53 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.587+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:84 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63335c00 con 0x55bf630c8800 2024-09-13T23:43:51.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.587+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(84 gc.17 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.587+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:85 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334000 con 0x55bf630c8800 2024-09-13T23:43:51.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.591+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(85 gc.17 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.591+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:86 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334400 con 0x55bf630c8800 2024-09-13T23:43:51.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.591+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(86 gc.17 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.591+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:87 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334800 con 0x55bf630cb000 2024-09-13T23:43:51.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.591+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 43 ==== osd_op_reply(87 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.591+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:88 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334c00 con 0x55bf630cb000 2024-09-13T23:43:51.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.591+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 44 ==== osd_op_reply(88 gc.18 [call out=24b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.591+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:89 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289800 con 0x55bf630cb000 2024-09-13T23:43:51.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.595+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 45 ==== osd_op_reply(89 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.595+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:90 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288c00 con 0x55bf63135800 2024-09-13T23:43:51.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.595+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(90 gc.19 [call out=15b] v0'0 uv164 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.595+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:91 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289000 con 0x55bf63135800 2024-09-13T23:43:51.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.595+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(91 gc.19 [call out=24b] v0'0 uv164 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.595+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:92 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289400 con 0x55bf63135800 2024-09-13T23:43:51.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.595+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(92 gc.19 [call out=15b] v0'0 uv164 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.595+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:93 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288400 con 0x55bf630cb000 2024-09-13T23:43:51.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.595+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 46 ==== osd_op_reply(93 gc.20 [call out=15b] v0'0 uv47 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.595+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:94 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289c00 con 0x55bf630cb000 2024-09-13T23:43:51.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.599+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 47 ==== osd_op_reply(94 gc.20 [call out=24b] v0'0 uv47 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.599+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:95 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288800 con 0x55bf630cb000 2024-09-13T23:43:51.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.599+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 48 ==== osd_op_reply(95 gc.20 [call out=15b] v0'0 uv47 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.599+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:96 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630ca400 con 0x55bf630cb000 2024-09-13T23:43:51.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.599+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 49 ==== osd_op_reply(96 gc.21 [call out=15b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.599+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:97 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63135c00 con 0x55bf630cb000 2024-09-13T23:43:51.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.599+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 50 ==== osd_op_reply(97 gc.21 [call out=24b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.599+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:98 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9800 con 0x55bf630cb000 2024-09-13T23:43:51.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.603+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 51 ==== osd_op_reply(98 gc.21 [call out=15b] v0'0 uv31 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.603+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:99 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9400 con 0x55bf630c8800 2024-09-13T23:43:51.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.603+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(99 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.603+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:100 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630ca400 con 0x55bf630c8800 2024-09-13T23:43:51.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.603+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(100 gc.22 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.603+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:101 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb800 con 0x55bf630c8800 2024-09-13T23:43:51.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.603+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(101 gc.22 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.603+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:102 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb400 con 0x55bf630cb000 2024-09-13T23:43:51.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.603+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 52 ==== osd_op_reply(102 gc.23 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.603+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:103 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289800 con 0x55bf630cb000 2024-09-13T23:43:51.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.607+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 53 ==== osd_op_reply(103 gc.23 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.607+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:104 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288c00 con 0x55bf630cb000 2024-09-13T23:43:51.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.607+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 54 ==== osd_op_reply(104 gc.23 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.607+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:105 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289000 con 0x55bf630c8800 2024-09-13T23:43:51.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.607+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== osd_op_reply(105 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.607+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:106 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289400 con 0x55bf630c8800 2024-09-13T23:43:51.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.607+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 25 ==== osd_op_reply(106 gc.24 [call out=24b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.607+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:107 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288400 con 0x55bf630c8800 2024-09-13T23:43:51.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.611+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 26 ==== osd_op_reply(107 gc.24 [call out=15b] v0'0 uv17 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.611+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:108 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289c00 con 0x55bf630c8800 2024-09-13T23:43:51.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.611+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 27 ==== osd_op_reply(108 gc.25 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.611+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:109 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288800 con 0x55bf630c8800 2024-09-13T23:43:51.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.611+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 28 ==== osd_op_reply(109 gc.25 [call out=24b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.611+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:110 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334000 con 0x55bf630c8800 2024-09-13T23:43:51.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.611+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 29 ==== osd_op_reply(110 gc.25 [call out=15b] v0'0 uv39 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.611+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:111 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334400 con 0x55bf630c8800 2024-09-13T23:43:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.611+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 30 ==== osd_op_reply(111 gc.26 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.611+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:112 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334800 con 0x55bf630c8800 2024-09-13T23:43:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.615+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 31 ==== osd_op_reply(112 gc.26 [call out=24b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.615+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:113 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334c00 con 0x55bf630c8800 2024-09-13T23:43:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.615+0000 7f1654875700 1 -- 172.21.15.167:0/995183761 <== osd.3 v2:172.21.15.167:6816/3688363524 32 ==== osd_op_reply(113 gc.26 [call out=15b] v0'0 uv23 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8800 2024-09-13T23:43:51.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.615+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:114 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63335000 con 0x55bf630cb000 2024-09-13T23:43:51.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.615+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 55 ==== osd_op_reply(114 gc.27 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.615+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:115 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63335400 con 0x55bf630cb000 2024-09-13T23:43:51.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.615+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 56 ==== osd_op_reply(115 gc.27 [call out=24b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.615+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:116 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63335800 con 0x55bf630cb000 2024-09-13T23:43:51.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.615+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 57 ==== osd_op_reply(116 gc.27 [call out=15b] v0'0 uv21 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.615+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:117 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63335c00 con 0x55bf63135800 2024-09-13T23:43:51.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(117 gc.28 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:118 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334000 con 0x55bf63135800 2024-09-13T23:43:51.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(118 gc.28 [call out=24b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:119 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289000 con 0x55bf63135800 2024-09-13T23:43:51.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(119 gc.28 [call out=15b] v0'0 uv37 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:120 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289400 con 0x55bf630cb000 2024-09-13T23:43:51.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 58 ==== osd_op_reply(120 gc.29 [call out=15b] v0'0 uv49 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:121 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288400 con 0x55bf630cb000 2024-09-13T23:43:51.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 59 ==== osd_op_reply(121 gc.29 [call out=24b] v0'0 uv49 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:122 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289c00 con 0x55bf630cb000 2024-09-13T23:43:51.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.0 v2:172.21.15.12:6800/2649399468 60 ==== osd_op_reply(122 gc.29 [call out=15b] v0'0 uv49 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf630cb000 2024-09-13T23:43:51.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:123 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288800 con 0x55bf63135800 2024-09-13T23:43:51.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(123 gc.30 [call out=15b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.619+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:124 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630ca400 con 0x55bf63135800 2024-09-13T23:43:51.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.623+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(124 gc.30 [call out=24b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.623+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:125 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630cb800 con 0x55bf63135800 2024-09-13T23:43:51.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.623+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 <== osd.1 v2:172.21.15.167:6800/3875661733 28 ==== osd_op_reply(125 gc.30 [call out=15b] v0'0 uv42 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135800 2024-09-13T23:43:51.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.623+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:126 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf630c9400 con 0x55bf630ca000 2024-09-13T23:43:51.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.623+0000 7f1653873700 1 -- 172.21.15.167:0/995183761 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(126 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630ca000 2024-09-13T23:43:51.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.623+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:127 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289800 con 0x55bf630ca000 2024-09-13T23:43:51.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.623+0000 7f1653873700 1 -- 172.21.15.167:0/995183761 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(127 gc.31 [call out=24b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+24 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630ca000 2024-09-13T23:43:51.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.623+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:128 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288c00 con 0x55bf630ca000 2024-09-13T23:43:51.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.623+0000 7f1653873700 1 -- 172.21.15.167:0/995183761 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(128 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) v8 ==== 149+0+15 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630ca000 2024-09-13T23:43:51.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.627+0000 7f163b042700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:51.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.627+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94280490164224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289000 con 0x55bf63135000 2024-09-13T23:43:51.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.631+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 data_loggenerations_metadata [watch unwatch cookie 94280490164224] v35'172 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.631+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94280488483584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289400 con 0x55bf63135400 2024-09-13T23:43:51.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.631+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(63 notify.0 [watch unwatch cookie 94280488483584] v35'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.631+0000 7f16558bbf40 20 remove_watcher() i=0 2024-09-13T23:43:51.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.631+0000 7f16558bbf40 2 removed watcher, disabling cache 2024-09-13T23:43:51.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.631+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94280488484736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63288400 con 0x55bf63135000 2024-09-13T23:43:51.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.635+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 notify.1 [watch unwatch cookie 94280488484736] v35'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf63135000 2024-09-13T23:43:51.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.635+0000 7f16558bbf40 20 remove_watcher() i=1 2024-09-13T23:43:51.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.635+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94280488485888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63289c00 con 0x55bf63135400 2024-09-13T23:43:51.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.639+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 notify.2 [watch unwatch cookie 94280488485888] v35'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.639+0000 7f16558bbf40 20 remove_watcher() i=2 2024-09-13T23:43:51.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.639+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94280488487040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63335000 con 0x55bf630c8400 2024-09-13T23:43:51.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.639+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(66 notify.3 [watch unwatch cookie 94280488487040] v35'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.639+0000 7f16558bbf40 20 remove_watcher() i=3 2024-09-13T23:43:51.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.639+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94280488488192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63335400 con 0x55bf63135400 2024-09-13T23:43:51.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.643+0000 7f1654074700 1 -- 172.21.15.167:0/3519934559 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(67 notify.4 [watch unwatch cookie 94280488488192] v35'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5200 con 0x55bf63135400 2024-09-13T23:43:51.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.643+0000 7f16558bbf40 20 remove_watcher() i=4 2024-09-13T23:43:51.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.643+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94280488489344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63335800 con 0x55bf630c8000 2024-09-13T23:43:51.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.647+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(68 notify.5 [watch unwatch cookie 94280488489344] v35'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.647+0000 7f16558bbf40 20 remove_watcher() i=5 2024-09-13T23:43:51.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.647+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94280488490496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63335c00 con 0x55bf630c8000 2024-09-13T23:43:51.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.647+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(69 notify.6 [watch unwatch cookie 94280488490496] v35'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e5440 con 0x55bf630c8000 2024-09-13T23:43:51.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.647+0000 7f16558bbf40 20 remove_watcher() i=6 2024-09-13T23:43:51.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.647+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94280488491648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bf63334000 con 0x55bf630c8400 2024-09-13T23:43:51.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f1654875700 1 -- 172.21.15.167:0/3519934559 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(70 notify.7 [watch unwatch cookie 94280488491648] v35'167 uv165 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bf631e4b40 con 0x55bf630c8400 2024-09-13T23:43:51.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 20 remove_watcher() i=7 2024-09-13T23:43:51.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bf63135400 msgr2=0x55bf63072100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bf63135400 0x55bf63072100 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bf63135000 msgr2=0x55bf63071b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bf63135000 0x55bf63071b80 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bf630c8400 msgr2=0x55bf63072c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bf630c8400 0x55bf63072c00 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bf630c8000 msgr2=0x55bf63072680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bf630c8000 0x55bf63072680 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63134c00 msgr2=0x55bf63071600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63134c00 0x55bf63071600 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x55bf6306b920 tx=0x55bf62e8f800 comp rx=0 tx=0).stop 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 msgr2=0x55bf63070b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 0x55bf63070b00 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x55bf63bf49c0 tx=0x55bf630ea840 comp rx=0 tx=0).stop 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f1653873700 1 -- 172.21.15.167:0/3519934559 reap_dead start 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 shutdown_connections 2024-09-13T23:43:51.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/3519934559 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b800 0x55bf63070b00 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 >> 172.21.15.167:0/3519934559 conn(0x55bf62d8b400 msgr2=0x55bf62e8b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 shutdown_connections 2024-09-13T23:43:51.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/3519934559 wait complete. 2024-09-13T23:43:51.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bf630cb000 msgr2=0x55bf63073700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bf630cb000 0x55bf63073700 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bf63135800 msgr2=0x55bf6328f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bf63135800 0x55bf6328f080 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bf630ca000 msgr2=0x55bf6328e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bf630ca000 0x55bf6328e580 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bf630c8800 msgr2=0x55bf6328eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bf630c8800 0x55bf6328eb00 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63288000 msgr2=0x55bf6328e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63288000 0x55bf6328e000 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55bf6313b6e0 tx=0x55bf630eb7a0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 msgr2=0x55bf63073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f1654074700 1 -- 172.21.15.167:0/995183761 reap_dead start 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.651+0000 7f16558bbf40 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 0x55bf63073180 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55bf6320a2d0 tx=0x55bf631dc420 comp rx=0 tx=0).stop 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 shutdown_connections 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 --2- 172.21.15.167:0/995183761 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf630cac00 0x55bf63073180 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 >> 172.21.15.167:0/995183761 conn(0x55bf630cbc00 msgr2=0x55bf62e8c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 shutdown_connections 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 -- 172.21.15.167:0/995183761 wait complete. 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 -- 172.21.15.167:0/866486522 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63134000 msgr2=0x55bf63071080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 --2- 172.21.15.167:0/866486522 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63134000 0x55bf63071080 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55bf63bf5050 tx=0x55bf630ea240 comp rx=0 tx=0).stop 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 -- 172.21.15.167:0/866486522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 msgr2=0x55bf63070580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 --2- 172.21.15.167:0/866486522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 0x55bf63070580 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x55bf63bf40f0 tx=0x55bf62e8fa40 comp rx=0 tx=0).stop 2024-09-13T23:43:51.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 -- 172.21.15.167:0/866486522 shutdown_connections 2024-09-13T23:43:51.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 --2- 172.21.15.167:0/866486522 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bf63134000 0x55bf63071080 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 --2- 172.21.15.167:0/866486522 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bf62d8b000 0x55bf63070580 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 -- 172.21.15.167:0/866486522 >> 172.21.15.167:0/866486522 conn(0x55bf62d8ac00 msgr2=0x55bf62e8b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 -- 172.21.15.167:0/866486522 shutdown_connections 2024-09-13T23:43:51.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.655+0000 7f16558bbf40 1 -- 172.21.15.167:0/866486522 wait complete. 2024-09-13T23:43:51.667 INFO:teuthology.orchestra.run.smithi167.stdout:[] 2024-09-13T23:43:51.669 DEBUG:tasks.util.rgw: json result: [] 2024-09-13T23:43:51.670 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-09-13T23:43:51.670 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-09-13T23:43:51.670 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-09-13T23:43:51.710 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:51.710 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:51.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea4b17f40 1 Processor -- start 2024-09-13T23:43:51.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea4b17f40 1 -- start start 2024-09-13T23:43:51.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea4b17f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f370d19000 0x55f370ffe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea4b17f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f370ddd500 con 0x55f370d19400 2024-09-13T23:43:51.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea4b17f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f370da6a00 con 0x55f370d19000 2024-09-13T23:43:51.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea32d0700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f370d19000 0x55f370ffe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea2acf700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55f370d19400 0x55f370e18800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:49328/0 (socket says 172.21.15.167:49328) 2024-09-13T23:43:51.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2374675257 learned_addr learned my addr 172.21.15.167:0/2374675257 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:51.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea22ce700 1 -- 172.21.15.167:0/2374675257 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845840971 0 0) 0x55f370e74000 con 0x55f370d19400 2024-09-13T23:43:51.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea22ce700 1 -- 172.21.15.167:0/2374675257 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x55f370ddd680 con 0x55f370d19400 2024-09-13T23:43:51.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2374675257 >> v1:172.21.15.12:6789/0 conn(0x55f370d19400 legacy=0x55f370e18800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.713 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2374675257 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f370d2bd40 con 0x55f370d19000 2024-09-13T23:43:51.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea32d0700 1 --2- 172.21.15.167:0/2374675257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19000 0x55f370ffe580 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55f370ff9a70 tx=0x55f370e1d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d867cc03ab6a73a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea22ce700 1 -- 172.21.15.167:0/2374675257 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f370dbd4a0 con 0x55f370d19000 2024-09-13T23:43:51.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea22ce700 1 -- 172.21.15.167:0/2374675257 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f370ddd500 con 0x55f370d19000 2024-09-13T23:43:51.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.703+0000 7f2ea22ce700 1 -- 172.21.15.167:0/2374675257 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f370dbd600 con 0x55f370d19000 2024-09-13T23:43:51.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2374675257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19000 msgr2=0x55f370ffe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2374675257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19000 0x55f370ffe580 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x55f370ff9a70 tx=0x55f370e1d980 comp rx=0 tx=0).stop 2024-09-13T23:43:51.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2374675257 shutdown_connections 2024-09-13T23:43:51.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2374675257 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19000 0x55f370ffe580 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2374675257 >> 172.21.15.167:0/2374675257 conn(0x55f370d18c00 msgr2=0x55f370e18000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2374675257 shutdown_connections 2024-09-13T23:43:51.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2374675257 wait complete. 2024-09-13T23:43:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 Processor -- start 2024-09-13T23:43:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 -- start start 2024-09-13T23:43:51.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19000 0x55f370ffe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f370da6500 con 0x55f370d19000 2024-09-13T23:43:51.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea32d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19000 0x55f370ffe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea32d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19000 0x55f370ffe580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52266/0 (socket says 172.21.15.167:52266) 2024-09-13T23:43:51.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea32d0700 1 -- 172.21.15.167:0/253414048 learned_addr learned my addr 172.21.15.167:0/253414048 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea32d0700 1 -- 172.21.15.167:0/253414048 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f370d2bba0 con 0x55f370d19000 2024-09-13T23:43:51.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea32d0700 1 --2- 172.21.15.167:0/253414048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19000 0x55f370ffe580 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x55f371ae3e00 tx=0x55f370e1d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1eb99585dcf869 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea1acd700 1 -- 172.21.15.167:0/253414048 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f370dbd760 con 0x55f370d19000 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea1acd700 1 -- 172.21.15.167:0/253414048 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f370ddd980 con 0x55f370d19000 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea1acd700 1 -- 172.21.15.167:0/253414048 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f370dbd4a0 con 0x55f370d19000 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/253414048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19000 msgr2=0x55f370ffe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/253414048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19000 0x55f370ffe580 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x55f371ae3e00 tx=0x55f370e1d800 comp rx=0 tx=0).stop 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/253414048 shutdown_connections 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/253414048 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19000 0x55f370ffe580 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/253414048 >> 172.21.15.167:0/253414048 conn(0x55f370d18c00 msgr2=0x55f370e19000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/253414048 shutdown_connections 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.707+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/253414048 wait complete. 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea4b17f40 1 Processor -- start 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea4b17f40 1 -- start start 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea4b17f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19400 0x55f370ffeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea4b17f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f370da63c0 con 0x55f370d19400 2024-09-13T23:43:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea32d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19400 0x55f370ffeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea32d0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19400 0x55f370ffeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52272/0 (socket says 172.21.15.167:52272) 2024-09-13T23:43:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea32d0700 1 -- 172.21.15.167:0/3513957366 learned_addr learned my addr 172.21.15.167:0/3513957366 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea32d0700 1 -- 172.21.15.167:0/3513957366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f370d2ba00 con 0x55f370d19400 2024-09-13T23:43:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea32d0700 1 --2- 172.21.15.167:0/3513957366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19400 0x55f370ffeb00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x55f371ae3f50 tx=0x55f370e1d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea02ca700 1 -- 172.21.15.167:0/3513957366 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f370dbd600 con 0x55f370d19400 2024-09-13T23:43:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea02ca700 1 -- 172.21.15.167:0/3513957366 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f370dddb00 con 0x55f370d19400 2024-09-13T23:43:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/3513957366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f370d2b860 con 0x55f370d19400 2024-09-13T23:43:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea02ca700 1 -- 172.21.15.167:0/3513957366 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f370dbd8c0 con 0x55f370d19400 2024-09-13T23:43:51.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/3513957366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f370d2b6c0 con 0x55f370d19400 2024-09-13T23:43:51.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea4b17f40 1 Processor -- start 2024-09-13T23:43:51.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.711+0000 7f2ea4b17f40 1 -- start start 2024-09-13T23:43:51.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea4b17f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 0x55f370fff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea4b17f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f370d3e640 con 0x55f370d19c00 2024-09-13T23:43:51.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea02ca700 1 -- 172.21.15.167:0/3513957366 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f370e12b00 con 0x55f370d19400 2024-09-13T23:43:51.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea3ad1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 0x55f370fff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea02ca700 1 --2- 172.21.15.167:0/3513957366 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3710c0000 0x55f371000100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea3ad1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 0x55f370fff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52284/0 (socket says 172.21.15.167:52284) 2024-09-13T23:43:51.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2491179345 learned_addr learned my addr 172.21.15.167:0/2491179345 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea02ca700 1 -- 172.21.15.167:0/3513957366 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f370d7f180 con 0x55f370d19400 2024-09-13T23:43:51.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea32d0700 1 --2- 172.21.15.167:0/3513957366 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3710c0000 0x55f371000100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2491179345 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f370d2a000 con 0x55f370d19c00 2024-09-13T23:43:51.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea32d0700 1 --2- 172.21.15.167:0/3513957366 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3710c0000 0x55f371000100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55f371041620 tx=0x55f371080b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea3ad1700 1 --2- 172.21.15.167:0/2491179345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 0x55f370fff080 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55f37109a780 tx=0x55f371080e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfce99f32a82ecd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2e9dac5700 1 -- 172.21.15.167:0/2491179345 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f370dbd600 con 0x55f370d19c00 2024-09-13T23:43:51.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2e9dac5700 1 -- 172.21.15.167:0/2491179345 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f370dddb00 con 0x55f370d19c00 2024-09-13T23:43:51.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2e9dac5700 1 -- 172.21.15.167:0/2491179345 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f370dbd8c0 con 0x55f370d19c00 2024-09-13T23:43:51.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2491179345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 msgr2=0x55f370fff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2491179345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 0x55f370fff080 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x55f37109a780 tx=0x55f371080e40 comp rx=0 tx=0).stop 2024-09-13T23:43:51.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2491179345 shutdown_connections 2024-09-13T23:43:51.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2491179345 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 0x55f370fff080 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2491179345 >> 172.21.15.167:0/2491179345 conn(0x55f370d19800 msgr2=0x55f370e19800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2491179345 shutdown_connections 2024-09-13T23:43:51.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.715+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2491179345 wait complete. 2024-09-13T23:43:51.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2ea4b17f40 1 Processor -- start 2024-09-13T23:43:51.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2ea4b17f40 1 -- start start 2024-09-13T23:43:51.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2ea4b17f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 0x55f370fff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2ea4b17f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f370d3e8c0 con 0x55f370d19c00 2024-09-13T23:43:51.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2ea3ad1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 0x55f370fff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2ea3ad1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 0x55f370fff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52286/0 (socket says 172.21.15.167:52286) 2024-09-13T23:43:51.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 learned_addr learned my addr 172.21.15.167:0/2009197521 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f370d2aea0 con 0x55f370d19c00 2024-09-13T23:43:51.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2ea3ad1700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 0x55f370fff080 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x55f371040630 tx=0x55f3710805a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2e9c2c2700 1 -- 172.21.15.167:0/2009197521 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f370dbd760 con 0x55f370d19c00 2024-09-13T23:43:51.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2e9c2c2700 1 -- 172.21.15.167:0/2009197521 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f3710f2000 con 0x55f370d19c00 2024-09-13T23:43:51.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f370d2a820 con 0x55f370d19c00 2024-09-13T23:43:51.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2e9c2c2700 1 -- 172.21.15.167:0/2009197521 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f370dbd600 con 0x55f370d19c00 2024-09-13T23:43:51.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.719+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f370d2a9c0 con 0x55f370d19c00 2024-09-13T23:43:51.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea4b17f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2e9c2c2700 1 -- 172.21.15.167:0/2009197521 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f370e12dc0 con 0x55f370d19c00 2024-09-13T23:43:51.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2e9c2c2700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3710c0c00 0x55f371000680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2e9c2c2700 1 -- 172.21.15.167:0/2009197521 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f370d7f340 con 0x55f370d19c00 2024-09-13T23:43:51.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f3710c1000 0x55f370fff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea2acf700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3710c0c00 0x55f371000680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f3710c1400 con 0x55f3710c1000 2024-09-13T23:43:51.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea3ad1700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f3710c1000 0x55f370fff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea2acf700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3710c0c00 0x55f371000680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55f371ae2030 tx=0x55f37117c0c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea3ad1700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f3710c1000 0x55f370fff600 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea4b17f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea4b17f40 20 realm 2024-09-13T23:43:51.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea4b17f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f3710c1800 con 0x55f3710c1000 2024-09-13T23:43:51.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea4b17f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea4b17f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:51.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea4b17f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.723+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f3710c1c00 con 0x55f3710c1000 2024-09-13T23:43:51.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea4b17f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea4b17f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f3710c1400 0x55f370fffb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f3710c1800 con 0x55f3710c1400 2024-09-13T23:43:51.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea32d0700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f3710c1400 0x55f370fffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea32d0700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f3710c1400 0x55f370fffb80 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea4b17f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:51.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea4b17f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f3710c1c00 con 0x55f3710c1400 2024-09-13T23:43:51.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea4b17f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:51.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea4b17f40 20 searching for the correct realm 2024-09-13T23:43:51.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f3710c1800 0x55f371000c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f3710c1c00 con 0x55f3710c1800 2024-09-13T23:43:51.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea2acf700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f3710c1800 0x55f371000c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.727+0000 7f2ea2acf700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f3710c1800 0x55f371000c00 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f3710c1c00 con 0x55f3710c1400 2024-09-13T23:43:51.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea32d0700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f3710c1c00 0x55f370ffe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1c00 2024-09-13T23:43:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea3ad1700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f3710c1c00 0x55f370ffe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea3ad1700 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f3710c1c00 0x55f370ffe580 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1000 2024-09-13T23:43:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1000 2024-09-13T23:43:51.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.731+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1800 2024-09-13T23:43:51.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1000 2024-09-13T23:43:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1400 2024-09-13T23:43:51.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1400 2024-09-13T23:43:51.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1000 2024-09-13T23:43:51.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1c00 2024-09-13T23:43:51.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1000 2024-09-13T23:43:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.735+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1c00 2024-09-13T23:43:51.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.739+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.739+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1400 2024-09-13T23:43:51.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.739+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.739+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1800 2024-09-13T23:43:51.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.739+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.739+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1000 2024-09-13T23:43:51.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.739+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.739+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1400 2024-09-13T23:43:51.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.739+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.739+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1800 2024-09-13T23:43:51.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.739+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.739+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1000 2024-09-13T23:43:51.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.743+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.743+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1c00 2024-09-13T23:43:51.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.743+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.743+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1000 2024-09-13T23:43:51.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.743+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.743+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1800 2024-09-13T23:43:51.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.743+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.743+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1c00 2024-09-13T23:43:51.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.743+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.743+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1000 2024-09-13T23:43:51.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.743+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.743+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1c00 2024-09-13T23:43:51.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.747+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.747+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1800 2024-09-13T23:43:51.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.747+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.747+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1c00 2024-09-13T23:43:51.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.747+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.747+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1400 2024-09-13T23:43:51.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.747+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.747+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1400 2024-09-13T23:43:51.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.747+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.747+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1800 2024-09-13T23:43:51.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.747+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.747+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1400 2024-09-13T23:43:51.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1400 2024-09-13T23:43:51.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:51.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:51.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:51.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:51.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c800 con 0x55f3710c1000 2024-09-13T23:43:51.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118cc00 con 0x55f3710c1400 2024-09-13T23:43:51.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:51.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118d000 con 0x55f3710c1400 2024-09-13T23:43:51.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:51.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 zone default found 2024-09-13T23:43:51.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 4 Realm: () 2024-09-13T23:43:51.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:51.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:51.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:51.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 zonegroup default 2024-09-13T23:43:51.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.751+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118d400 con 0x55f3710c1c00 2024-09-13T23:43:51.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea4b17f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea4b17f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118d800 con 0x55f3710c1800 2024-09-13T23:43:51.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea4b17f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea4b17f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118dc00 con 0x55f3710c1800 2024-09-13T23:43:51.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea4b17f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:51.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea4b17f40 20 started sync module instance, tier type = 2024-09-13T23:43:51.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea4b17f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:51.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.755+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118d800 con 0x55f3710c1400 2024-09-13T23:43:51.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.759+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.759+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94504061197056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118dc00 con 0x55f3710c1400 2024-09-13T23:43:51.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.759+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c400 con 0x55f3710c1000 2024-09-13T23:43:51.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.759+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.759+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94504061198208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1000 2024-09-13T23:43:51.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.759+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118cc00 con 0x55f3710c1400 2024-09-13T23:43:51.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.759+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94504061197056] v35'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.763+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94504061198208] v35'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.763+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.763+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94504061199360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118d000 con 0x55f3710c1400 2024-09-13T23:43:51.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.763+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c800 con 0x55f3710c1c00 2024-09-13T23:43:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.767+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.767+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94504061200512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118d400 con 0x55f3710c1c00 2024-09-13T23:43:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.767+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94504061199360] v35'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.767+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c400 con 0x55f3710c1400 2024-09-13T23:43:51.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.767+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94504061200512] v35'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.771+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.771+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94504061201664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1400 2024-09-13T23:43:51.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.771+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118d800 con 0x55f3710c1800 2024-09-13T23:43:51.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.771+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.771+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94504061202816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118dc00 con 0x55f3710c1800 2024-09-13T23:43:51.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.771+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118cc00 con 0x55f3710c1800 2024-09-13T23:43:51.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.771+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94504061201664] v35'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94504061202816] v35'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94504061203968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118d000 con 0x55f3710c1800 2024-09-13T23:43:51.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c400 con 0x55f3710c1c00 2024-09-13T23:43:51.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94504061203968] v35'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'168 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94504061205120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118d800 con 0x55f3710c1c00 2024-09-13T23:43:51.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea4b17f40 20 add_watcher() i=0 2024-09-13T23:43:51.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea4b17f40 20 add_watcher() i=1 2024-09-13T23:43:51.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea4b17f40 20 add_watcher() i=2 2024-09-13T23:43:51.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea4b17f40 20 add_watcher() i=3 2024-09-13T23:43:51.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea4b17f40 20 add_watcher() i=4 2024-09-13T23:43:51.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea4b17f40 20 add_watcher() i=5 2024-09-13T23:43:51.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.775+0000 7f2ea4b17f40 20 add_watcher() i=6 2024-09-13T23:43:51.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94504061205120] v35'169 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2ea4b17f40 20 add_watcher() i=7 2024-09-13T23:43:51.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2ea4b17f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:51.786 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118dc00 con 0x55f3710c1000 2024-09-13T23:43:51.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94504062918656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118cc00 con 0x55f3710c1000 2024-09-13T23:43:51.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94504062918656] v35'173 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2e8a29e700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:51.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2ea4b17f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:51.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2ea4b17f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:51.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2ea4b17f40 1 Processor -- start 2024-09-13T23:43:51.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2ea4b17f40 1 -- start start 2024-09-13T23:43:51.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.779+0000 7f2ea4b17f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 0x55f371001180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea4b17f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f370da63c0 con 0x55f37118d400 2024-09-13T23:43:51.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea2acf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 0x55f371001180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea2acf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 0x55f371001180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52288/0 (socket says 172.21.15.167:52288) 2024-09-13T23:43:51.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea2acf700 1 -- 172.21.15.167:0/1449626856 learned_addr learned my addr 172.21.15.167:0/1449626856 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea2acf700 1 -- 172.21.15.167:0/1449626856 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f370d2ab60 con 0x55f37118d400 2024-09-13T23:43:51.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea2acf700 1 --2- 172.21.15.167:0/1449626856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 0x55f371001180 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x55f3711a40c0 tx=0x55f37117c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b89b33fec89fb96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2e89a9d700 1 -- 172.21.15.167:0/1449626856 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f3711a8000 con 0x55f37118d400 2024-09-13T23:43:51.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2e89a9d700 1 -- 172.21.15.167:0/1449626856 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f3710f2480 con 0x55f37118d400 2024-09-13T23:43:51.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2e89a9d700 1 -- 172.21.15.167:0/1449626856 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f3711a8160 con 0x55f37118d400 2024-09-13T23:43:51.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/1449626856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 msgr2=0x55f371001180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/1449626856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 0x55f371001180 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x55f3711a40c0 tx=0x55f37117c060 comp rx=0 tx=0).stop 2024-09-13T23:43:51.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/1449626856 shutdown_connections 2024-09-13T23:43:51.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/1449626856 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 0x55f371001180 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/1449626856 >> 172.21.15.167:0/1449626856 conn(0x55f37118c800 msgr2=0x55f370e1a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/1449626856 shutdown_connections 2024-09-13T23:43:51.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/1449626856 wait complete. 2024-09-13T23:43:51.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea4b17f40 1 Processor -- start 2024-09-13T23:43:51.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea4b17f40 1 -- start start 2024-09-13T23:43:51.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea4b17f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 0x55f371001180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea4b17f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f370da6500 con 0x55f37118d400 2024-09-13T23:43:51.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea2acf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 0x55f371001180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea2acf700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 0x55f371001180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52300/0 (socket says 172.21.15.167:52300) 2024-09-13T23:43:51.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.783+0000 7f2ea2acf700 1 -- 172.21.15.167:0/328575789 learned_addr learned my addr 172.21.15.167:0/328575789 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:51.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2ea2acf700 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f370d2ad00 con 0x55f37118d400 2024-09-13T23:43:51.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2ea2acf700 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 0x55f371001180 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55f3711a5380 tx=0x55f37117c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2e8829a700 1 -- 172.21.15.167:0/328575789 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f3711a82c0 con 0x55f37118d400 2024-09-13T23:43:51.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2e8829a700 1 -- 172.21.15.167:0/328575789 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f3710f2600 con 0x55f37118d400 2024-09-13T23:43:51.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f370d2a340 con 0x55f37118d400 2024-09-13T23:43:51.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2e8829a700 1 -- 172.21.15.167:0/328575789 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f3711a8000 con 0x55f37118d400 2024-09-13T23:43:51.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f370d2ab60 con 0x55f37118d400 2024-09-13T23:43:51.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2e8829a700 1 -- 172.21.15.167:0/328575789 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f370e13b80 con 0x55f37118d400 2024-09-13T23:43:51.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2e8829a700 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f371220000 0x55f371226000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2e8829a700 1 -- 172.21.15.167:0/328575789 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f370d7f500 con 0x55f37118d400 2024-09-13T23:43:51.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2ea3ad1700 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f371220000 0x55f371226000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f37118c400 0x55f371001700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118d800 con 0x55f37118c400 2024-09-13T23:43:51.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2ea32d0700 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f37118c400 0x55f371001700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2ea3ad1700 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f371220000 0x55f371226000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55f3711140f0 tx=0x55f371081500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.787+0000 7f2ea32d0700 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f37118c400 0x55f371001700 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.791+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v35'35 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.791+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f37118dc00 0x55f371226580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.791+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371220400 con 0x55f37118dc00 2024-09-13T23:43:51.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.791+0000 7f2ea2acf700 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f37118dc00 0x55f371226580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.791+0000 7f2ea2acf700 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f37118dc00 0x55f371226580 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.795+0000 7f2ea2acf700 1 -- 172.21.15.167:0/328575789 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v35'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55f371135440 con 0x55f37118dc00 2024-09-13T23:43:51.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.795+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f371220800 0x55f371226b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.795+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371220c00 con 0x55f371220800 2024-09-13T23:43:51.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.795+0000 7f2ea3ad1700 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f371220800 0x55f371226b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.795+0000 7f2ea3ad1700 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f371220800 0x55f371226b00 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.799+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/328575789 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v35'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f371220800 2024-09-13T23:43:51.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.799+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f371221000 0x55f371227080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:51.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.799+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371221400 con 0x55f371221000 2024-09-13T23:43:51.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.799+0000 7f2ea32d0700 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f371221000 0x55f371227080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:51.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.799+0000 7f2ea32d0700 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f371221000 0x55f371227080 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:51.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.799+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v35'174 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f371221000 2024-09-13T23:43:51.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.799+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371221800 con 0x55f37118c400 2024-09-13T23:43:51.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.803+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v35'59 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.803+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371221c00 con 0x55f371220800 2024-09-13T23:43:51.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.807+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/328575789 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v35'42 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f371220800 2024-09-13T23:43:51.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.807+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254000 con 0x55f37118c400 2024-09-13T23:43:51.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.807+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v35'25 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.807+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254400 con 0x55f371221000 2024-09-13T23:43:51.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.811+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v35'40 uv35 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f371221000 2024-09-13T23:43:51.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.811+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254800 con 0x55f37118c400 2024-09-13T23:43:51.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.815+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v35'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.815+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254c00 con 0x55f37118c400 2024-09-13T23:43:51.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.815+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v35'30 uv28 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.815+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255000 con 0x55f37118c400 2024-09-13T23:43:51.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.819+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v35'60 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.819+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255400 con 0x55f371220800 2024-09-13T23:43:51.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.823+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/328575789 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v35'26 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f371220800 2024-09-13T23:43:51.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.823+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255800 con 0x55f37118c400 2024-09-13T23:43:51.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.823+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v35'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.823+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255c00 con 0x55f371221000 2024-09-13T23:43:51.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.827+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v35'45 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f371221000 2024-09-13T23:43:51.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.827+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254000 con 0x55f371221000 2024-09-13T23:43:51.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.831+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v35'175 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f371221000 2024-09-13T23:43:51.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.831+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254400 con 0x55f37118c400 2024-09-13T23:43:51.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.831+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v35'53 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.831+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371221400 con 0x55f37118c400 2024-09-13T23:43:51.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.835+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v35'61 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.835+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371221800 con 0x55f371220800 2024-09-13T23:43:51.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.839+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/328575789 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v35'25 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f371220800 2024-09-13T23:43:51.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.839+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f37118c400 2024-09-13T23:43:51.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.839+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v35'31 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.839+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118d800 con 0x55f371221000 2024-09-13T23:43:51.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.843+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v35'176 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f371221000 2024-09-13T23:43:51.852 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.843+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254800 con 0x55f37118c400 2024-09-13T23:43:51.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.847+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v35'54 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.855 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.847+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254c00 con 0x55f37118c400 2024-09-13T23:43:51.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.847+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v35'33 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.847+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255000 con 0x55f371220800 2024-09-13T23:43:51.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.851+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/328575789 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v35'25 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f371220800 2024-09-13T23:43:51.860 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.851+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255800 con 0x55f37118c400 2024-09-13T23:43:51.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.855+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v35'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.855+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255c00 con 0x55f371220800 2024-09-13T23:43:51.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.855+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/328575789 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v35'19 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f371220800 2024-09-13T23:43:51.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.855+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254000 con 0x55f371220800 2024-09-13T23:43:51.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.859+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/328575789 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v35'43 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f371220800 2024-09-13T23:43:51.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.859+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254400 con 0x55f371220800 2024-09-13T23:43:51.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.859+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/328575789 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v35'25 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f371220800 2024-09-13T23:43:51.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.863+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255400 con 0x55f37118c400 2024-09-13T23:43:51.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.863+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v35'23 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.863+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255000 con 0x55f371221000 2024-09-13T23:43:51.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.867+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v35'41 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f371221000 2024-09-13T23:43:51.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.867+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371220c00 con 0x55f37118c400 2024-09-13T23:43:51.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.871+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v35'55 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f37118c400 2024-09-13T23:43:51.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.871+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371221c00 con 0x55f371221000 2024-09-13T23:43:51.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.871+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v35'46 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f371221000 2024-09-13T23:43:51.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.871+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371221800 con 0x55f37118dc00 2024-09-13T23:43:51.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.875+0000 7f2ea2acf700 1 -- 172.21.15.167:0/328575789 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v35'29 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55f371135440 con 0x55f37118dc00 2024-09-13T23:43:51.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2e85a95700 20 reqs_thread_entry: start 2024-09-13T23:43:51.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2ea4b17f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:51.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2ea4b17f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:51.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2e83a91700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:22 2024 2024-09-13T23:43:51.888 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:51.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2e8326f700 20 reqs_thread_entry: start 2024-09-13T23:43:51.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2ea4b17f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:51.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2ea4b17f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:51.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254c00 con 0x55f3710c1000 2024-09-13T23:43:51.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv13 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2ea4b17f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:51.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2ea4b17f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:51.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2ea4b17f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:51.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2ea4b17f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:51.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255800 con 0x55f3710c1c00 2024-09-13T23:43:51.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.879+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.890 INFO:teuthology.orchestra.run.smithi167.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-09-13T23:43:51.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.883+0000 7f2e8a29e700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:51.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.883+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94504062918656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118c000 con 0x55f3710c1000 2024-09-13T23:43:51.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.883+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94504062918656] v35'177 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.883+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94504061197056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f37118d800 con 0x55f3710c1400 2024-09-13T23:43:51.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.887+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94504061197056] v35'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.887+0000 7f2ea4b17f40 20 remove_watcher() i=0 2024-09-13T23:43:51.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.887+0000 7f2ea4b17f40 2 removed watcher, disabling cache 2024-09-13T23:43:51.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.887+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94504061198208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371221400 con 0x55f3710c1000 2024-09-13T23:43:51.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.891+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94504061198208] v35'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1000 2024-09-13T23:43:51.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.891+0000 7f2ea4b17f40 20 remove_watcher() i=1 2024-09-13T23:43:51.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.891+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94504061199360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255c00 con 0x55f3710c1400 2024-09-13T23:43:51.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.891+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94504061199360] v35'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.891+0000 7f2ea4b17f40 20 remove_watcher() i=2 2024-09-13T23:43:51.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.891+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94504061200512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254000 con 0x55f3710c1c00 2024-09-13T23:43:51.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.895+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94504061200512] v35'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.895+0000 7f2ea4b17f40 20 remove_watcher() i=3 2024-09-13T23:43:51.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.895+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94504061201664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254400 con 0x55f3710c1400 2024-09-13T23:43:51.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.899+0000 7f2ea32d0700 1 -- 172.21.15.167:0/2009197521 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94504061201664] v35'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135200 con 0x55f3710c1400 2024-09-13T23:43:51.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.899+0000 7f2ea4b17f40 20 remove_watcher() i=4 2024-09-13T23:43:51.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.899+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94504061202816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371254c00 con 0x55f3710c1800 2024-09-13T23:43:51.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.899+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94504061202816] v35'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.899+0000 7f2ea4b17f40 20 remove_watcher() i=5 2024-09-13T23:43:51.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.899+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94504061203968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255800 con 0x55f3710c1800 2024-09-13T23:43:51.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.903+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94504061203968] v35'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371135440 con 0x55f3710c1800 2024-09-13T23:43:51.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.903+0000 7f2ea4b17f40 20 remove_watcher() i=6 2024-09-13T23:43:51.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.903+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94504061205120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f371255000 con 0x55f3710c1c00 2024-09-13T23:43:51.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea3ad1700 1 -- 172.21.15.167:0/2009197521 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94504061205120] v35'170 uv168 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f371134b40 con 0x55f3710c1c00 2024-09-13T23:43:51.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 20 remove_watcher() i=7 2024-09-13T23:43:51.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f3710c1400 msgr2=0x55f370fffb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f3710c1400 0x55f370fffb80 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f3710c1000 msgr2=0x55f370fff600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f3710c1000 0x55f370fff600 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f3710c1c00 msgr2=0x55f370ffe580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f3710c1c00 0x55f370ffe580 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f3710c1800 msgr2=0x55f371000c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f3710c1800 0x55f371000c00 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3710c0c00 msgr2=0x55f371000680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3710c0c00 0x55f371000680 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55f371ae2030 tx=0x55f37117c0c0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 msgr2=0x55f370fff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/2009197521 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19c00 0x55f370fff080 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x55f371040630 tx=0x55f3710805a0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 reap_dead start 2024-09-13T23:43:51.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea2acf700 1 -- 172.21.15.167:0/2009197521 reap_dead start 2024-09-13T23:43:51.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 shutdown_connections 2024-09-13T23:43:51.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 >> 172.21.15.167:0/2009197521 conn(0x55f370d19800 msgr2=0x55f370e19800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 shutdown_connections 2024-09-13T23:43:51.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/2009197521 wait complete. 2024-09-13T23:43:51.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f37118c400 msgr2=0x55f371001700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f37118c400 0x55f371001700 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f371221000 msgr2=0x55f371227080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f371221000 0x55f371227080 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f37118dc00 msgr2=0x55f371226580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f37118dc00 0x55f371226580 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f371220800 msgr2=0x55f371226b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f371220800 0x55f371226b00 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f371220000 msgr2=0x55f371226000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f371220000 0x55f371226000 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55f3711140f0 tx=0x55f371081500 comp rx=0 tx=0).stop 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea32d0700 1 -- 172.21.15.167:0/328575789 reap_dead start 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 msgr2=0x55f371001180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 0x55f371001180 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x55f3711a5380 tx=0x55f37117c180 comp rx=0 tx=0).stop 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 shutdown_connections 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/328575789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f37118d400 0x55f371001180 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 >> 172.21.15.167:0/328575789 conn(0x55f37118c800 msgr2=0x55f370e1a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 shutdown_connections 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/328575789 wait complete. 2024-09-13T23:43:51.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/3513957366 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3710c0000 msgr2=0x55f371000100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/3513957366 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3710c0000 0x55f371000100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x55f371041620 tx=0x55f371080b40 comp rx=0 tx=0).stop 2024-09-13T23:43:51.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/3513957366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19400 msgr2=0x55f370ffeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:51.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/3513957366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19400 0x55f370ffeb00 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x55f371ae3f50 tx=0x55f370e1d980 comp rx=0 tx=0).stop 2024-09-13T23:43:51.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/3513957366 shutdown_connections 2024-09-13T23:43:51.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/3513957366 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f3710c0000 0x55f371000100 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 --2- 172.21.15.167:0/3513957366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f370d19400 0x55f370ffeb00 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:51.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/3513957366 >> 172.21.15.167:0/3513957366 conn(0x55f370d18c00 msgr2=0x55f370e19000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:51.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/3513957366 shutdown_connections 2024-09-13T23:43:51.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:51.907+0000 7f2ea4b17f40 1 -- 172.21.15.167:0/3513957366 wait complete. 2024-09-13T23:43:51.923 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-09-13T23:43:51.938 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=236 failed=True 2024-09-13T23:43:51.947 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:52.016 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:52.032 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:52.077 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-09-13T23:43:52.088 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:52.089 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-09-13T23:43:52.089 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-09-13T23:43:52.089 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-09-13T23:43:52.128 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:52.128 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:52.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.119+0000 7fda7e9d4f40 1 Processor -- start 2024-09-13T23:43:52.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.119+0000 7fda7e9d4f40 1 -- start start 2024-09-13T23:43:52.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.119+0000 7fda7e9d4f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c749e83400 0x55c74a168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.119+0000 7fda7e9d4f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c749f47500 con 0x55c749e83000 2024-09-13T23:43:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.119+0000 7fda7e9d4f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c749f10a00 con 0x55c749e83400 2024-09-13T23:43:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.119+0000 7fda7d18d700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55c749e83000 0x55c749f82800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:49340/0 (socket says 172.21.15.167:49340) 2024-09-13T23:43:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.119+0000 7fda7d18d700 1 -- 172.21.15.167:0/3951799835 learned_addr learned my addr 172.21.15.167:0/3951799835 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:52.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.119+0000 7fda7c98c700 1 --2- 172.21.15.167:0/3951799835 >> v2:172.21.15.12:3300/0 conn(0x55c749e83400 0x55c74a168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7c98c700 1 -- 172.21.15.167:0/3951799835 >> v1:172.21.15.12:6789/0 conn(0x55c749e83000 legacy=0x55c749f82800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7c98c700 1 -- 172.21.15.167:0/3951799835 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c749e95d40 con 0x55c749e83400 2024-09-13T23:43:52.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7c98c700 1 --2- 172.21.15.167:0/3951799835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83400 0x55c74a168580 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x55c74a163920 tx=0x55c749f87920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad903ac4f9db926 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7c18b700 1 -- 172.21.15.167:0/3951799835 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c749f274a0 con 0x55c749e83400 2024-09-13T23:43:52.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7c18b700 1 -- 172.21.15.167:0/3951799835 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c749f47800 con 0x55c749e83400 2024-09-13T23:43:52.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7c18b700 1 -- 172.21.15.167:0/3951799835 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c749f27600 con 0x55c749e83400 2024-09-13T23:43:52.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/3951799835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83400 msgr2=0x55c74a168580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/3951799835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83400 0x55c74a168580 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x55c74a163920 tx=0x55c749f87920 comp rx=0 tx=0).stop 2024-09-13T23:43:52.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/3951799835 shutdown_connections 2024-09-13T23:43:52.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/3951799835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83400 0x55c74a168580 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/3951799835 >> 172.21.15.167:0/3951799835 conn(0x55c749e82c00 msgr2=0x55c749f82000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/3951799835 shutdown_connections 2024-09-13T23:43:52.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/3951799835 wait complete. 2024-09-13T23:43:52.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7e9d4f40 1 Processor -- start 2024-09-13T23:43:52.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7e9d4f40 1 -- start start 2024-09-13T23:43:52.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7e9d4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 0x55c74a168580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7e9d4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749f10500 con 0x55c749e83000 2024-09-13T23:43:52.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7d18d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 0x55c74a168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7d18d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 0x55c74a168580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52314/0 (socket says 172.21.15.167:52314) 2024-09-13T23:43:52.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7d18d700 1 -- 172.21.15.167:0/2665831038 learned_addr learned my addr 172.21.15.167:0/2665831038 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7d18d700 1 -- 172.21.15.167:0/2665831038 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c749e95ba0 con 0x55c749e83000 2024-09-13T23:43:52.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.123+0000 7fda7d18d700 1 --2- 172.21.15.167:0/2665831038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 0x55c74a168580 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55c74a162b10 tx=0x55c749f87b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f5fa6d5e4081a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7b98a700 1 -- 172.21.15.167:0/2665831038 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c749f274a0 con 0x55c749e83000 2024-09-13T23:43:52.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7b98a700 1 -- 172.21.15.167:0/2665831038 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c749f47b00 con 0x55c749e83000 2024-09-13T23:43:52.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7b98a700 1 -- 172.21.15.167:0/2665831038 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c749f27600 con 0x55c749e83000 2024-09-13T23:43:52.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2665831038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 msgr2=0x55c74a168580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/2665831038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 0x55c74a168580 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x55c74a162b10 tx=0x55c749f87b60 comp rx=0 tx=0).stop 2024-09-13T23:43:52.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2665831038 shutdown_connections 2024-09-13T23:43:52.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/2665831038 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 0x55c74a168580 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2665831038 >> 172.21.15.167:0/2665831038 conn(0x55c749e82c00 msgr2=0x55c749f83000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2665831038 shutdown_connections 2024-09-13T23:43:52.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2665831038 wait complete. 2024-09-13T23:43:52.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 Processor -- start 2024-09-13T23:43:52.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 -- start start 2024-09-13T23:43:52.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 0x55c74a168580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749f103c0 con 0x55c749e83000 2024-09-13T23:43:52.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7d18d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 0x55c74a168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7d18d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 0x55c74a168580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52336/0 (socket says 172.21.15.167:52336) 2024-09-13T23:43:52.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7d18d700 1 -- 172.21.15.167:0/2402376867 learned_addr learned my addr 172.21.15.167:0/2402376867 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7d18d700 1 -- 172.21.15.167:0/2402376867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c749e95a00 con 0x55c749e83000 2024-09-13T23:43:52.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7d18d700 1 --2- 172.21.15.167:0/2402376867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 0x55c74a168580 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55c749fdff80 tx=0x55c749f87a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7a187700 1 -- 172.21.15.167:0/2402376867 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c749f278c0 con 0x55c749e83000 2024-09-13T23:43:52.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7a187700 1 -- 172.21.15.167:0/2402376867 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c749f47c80 con 0x55c749e83000 2024-09-13T23:43:52.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2402376867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c749e95860 con 0x55c749e83000 2024-09-13T23:43:52.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7a187700 1 -- 172.21.15.167:0/2402376867 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c749f274a0 con 0x55c749e83000 2024-09-13T23:43:52.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.127+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2402376867 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c749e956c0 con 0x55c749e83000 2024-09-13T23:43:52.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7e9d4f40 1 Processor -- start 2024-09-13T23:43:52.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7e9d4f40 1 -- start start 2024-09-13T23:43:52.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7a187700 1 -- 172.21.15.167:0/2402376867 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c749f7cb00 con 0x55c749e83000 2024-09-13T23:43:52.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7a187700 1 --2- 172.21.15.167:0/2402376867 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a226000 0x55c74a169080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7a187700 1 -- 172.21.15.167:0/2402376867 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55c749ee9180 con 0x55c749e83000 2024-09-13T23:43:52.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7e9d4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 0x55c74a168b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7e9d4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749ea8640 con 0x55c749e83800 2024-09-13T23:43:52.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7d98e700 1 --2- 172.21.15.167:0/2402376867 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a226000 0x55c74a169080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7d18d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 0x55c74a168b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7d18d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 0x55c74a168b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52338/0 (socket says 172.21.15.167:52338) 2024-09-13T23:43:52.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7d18d700 1 -- 172.21.15.167:0/669523008 learned_addr learned my addr 172.21.15.167:0/669523008 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7d98e700 1 --2- 172.21.15.167:0/2402376867 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a226000 0x55c74a169080 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55c74a1a62d0 tx=0x55c74a1dede0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7d18d700 1 -- 172.21.15.167:0/669523008 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c749e94000 con 0x55c749e83800 2024-09-13T23:43:52.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda7d18d700 1 --2- 172.21.15.167:0/669523008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 0x55c74a168b00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55c74a1a7020 tx=0x55c74a1de240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6883c20b22636ba1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda77982700 1 -- 172.21.15.167:0/669523008 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c749f27600 con 0x55c749e83800 2024-09-13T23:43:52.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda77982700 1 -- 172.21.15.167:0/669523008 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c749f47c80 con 0x55c749e83800 2024-09-13T23:43:52.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.131+0000 7fda77982700 1 -- 172.21.15.167:0/669523008 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c749f27a20 con 0x55c749e83800 2024-09-13T23:43:52.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/669523008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 msgr2=0x55c74a168b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.143 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/669523008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 0x55c74a168b00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x55c74a1a7020 tx=0x55c74a1de240 comp rx=0 tx=0).stop 2024-09-13T23:43:52.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/669523008 shutdown_connections 2024-09-13T23:43:52.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/669523008 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 0x55c74a168b00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/669523008 >> 172.21.15.167:0/669523008 conn(0x55c749e83400 msgr2=0x55c749f83800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/669523008 shutdown_connections 2024-09-13T23:43:52.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/669523008 wait complete. 2024-09-13T23:43:52.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7e9d4f40 1 Processor -- start 2024-09-13T23:43:52.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7e9d4f40 1 -- start start 2024-09-13T23:43:52.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7e9d4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 0x55c74a168b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7e9d4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749ea88c0 con 0x55c749e83800 2024-09-13T23:43:52.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7d18d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 0x55c74a168b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7d18d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 0x55c74a168b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52350/0 (socket says 172.21.15.167:52350) 2024-09-13T23:43:52.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 learned_addr learned my addr 172.21.15.167:0/1381252636 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c749e94ea0 con 0x55c749e83800 2024-09-13T23:43:52.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7d18d700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 0x55c74a168b00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55c74a1a6570 tx=0x55c74a1defc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7617f700 1 -- 172.21.15.167:0/1381252636 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c749f27600 con 0x55c749e83800 2024-09-13T23:43:52.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7617f700 1 -- 172.21.15.167:0/1381252636 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c74a246000 con 0x55c749e83800 2024-09-13T23:43:52.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c749e94820 con 0x55c749e83800 2024-09-13T23:43:52.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.135+0000 7fda7617f700 1 -- 172.21.15.167:0/1381252636 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c749f27a20 con 0x55c749e83800 2024-09-13T23:43:52.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c749e949c0 con 0x55c749e83800 2024-09-13T23:43:52.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7617f700 1 -- 172.21.15.167:0/1381252636 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c749f7cdc0 con 0x55c749e83800 2024-09-13T23:43:52.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7617f700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a227000 0x55c74a169600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7617f700 1 -- 172.21.15.167:0/1381252636 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55c749ee9340 con 0x55c749e83800 2024-09-13T23:43:52.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7c98c700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a227000 0x55c74a169600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7c98c700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a227000 0x55c74a169600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55c74a163920 tx=0x55c749f87800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c74a227400 0x55c74a169b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a227800 con 0x55c74a227400 2024-09-13T23:43:52.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7d98e700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c74a227400 0x55c74a169b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7d98e700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c74a227400 0x55c74a169b80 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7e9d4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7e9d4f40 20 realm 2024-09-13T23:43:52.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.139+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a227c00 con 0x55c74a227400 2024-09-13T23:43:52.151 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:52.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a227800 con 0x55c74a227400 2024-09-13T23:43:52.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c74a227c00 0x55c74a16a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8000 con 0x55c74a227c00 2024-09-13T23:43:52.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7d18d700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c74a227c00 0x55c74a16a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7d18d700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c74a227c00 0x55c74a16a100 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:52.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227c00 2024-09-13T23:43:52.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:52.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 20 searching for the correct realm 2024-09-13T23:43:52.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c74a2b8800 0x55c74a16a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.143+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8c00 con 0x55c74a2b8800 2024-09-13T23:43:52.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7c98c700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c74a2b8800 0x55c74a16a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7c98c700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c74a2b8800 0x55c74a16a680 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8c00 con 0x55c74a227c00 2024-09-13T23:43:52.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7d18d700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c74a2b8000 0x55c74a16ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8000 2024-09-13T23:43:52.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7d98e700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c74a2b8000 0x55c74a16ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7d98e700 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c74a2b8000 0x55c74a16ac00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227400 2024-09-13T23:43:52.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.147+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227400 2024-09-13T23:43:52.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.151+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.151+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8800 2024-09-13T23:43:52.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.151+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.151+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227400 2024-09-13T23:43:52.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.151+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.151+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227c00 2024-09-13T23:43:52.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.151+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.151+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227c00 2024-09-13T23:43:52.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.151+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.151+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227400 2024-09-13T23:43:52.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.151+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.151+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8000 2024-09-13T23:43:52.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.155+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.155+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227400 2024-09-13T23:43:52.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.155+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.155+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8000 2024-09-13T23:43:52.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.155+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.155+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227c00 2024-09-13T23:43:52.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.155+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.155+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8800 2024-09-13T23:43:52.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.155+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.155+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227400 2024-09-13T23:43:52.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.155+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.155+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227c00 2024-09-13T23:43:52.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8800 2024-09-13T23:43:52.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227400 2024-09-13T23:43:52.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8000 2024-09-13T23:43:52.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227400 2024-09-13T23:43:52.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8800 2024-09-13T23:43:52.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8000 2024-09-13T23:43:52.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.159+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.163+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227400 2024-09-13T23:43:52.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.163+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.163+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8000 2024-09-13T23:43:52.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.163+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.163+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8800 2024-09-13T23:43:52.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.163+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.163+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8000 2024-09-13T23:43:52.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.163+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.163+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227c00 2024-09-13T23:43:52.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.163+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.163+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227c00 2024-09-13T23:43:52.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8800 2024-09-13T23:43:52.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227c00 2024-09-13T23:43:52.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227c00 2024-09-13T23:43:52.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7e9d4f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:52.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7e9d4f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:52.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7e9d4f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:52.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7e9d4f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:52.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9000 con 0x55c74a227400 2024-09-13T23:43:52.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7e9d4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.167+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9400 con 0x55c74a227c00 2024-09-13T23:43:52.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:52.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9800 con 0x55c74a227c00 2024-09-13T23:43:52.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:52.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 zone default found 2024-09-13T23:43:52.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 4 Realm: () 2024-09-13T23:43:52.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:52.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:52.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:52.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 zonegroup default 2024-09-13T23:43:52.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9c00 con 0x55c74a2b8000 2024-09-13T23:43:52.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9000 con 0x55c74a2b8800 2024-09-13T23:43:52.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9c00 con 0x55c74a2b8800 2024-09-13T23:43:52.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 started sync module instance, tier type = 2024-09-13T23:43:52.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.171+0000 7fda7e9d4f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:52.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.175+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a227800 con 0x55c74a227c00 2024-09-13T23:43:52.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.175+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.175+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94314429807360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8c00 con 0x55c74a227c00 2024-09-13T23:43:52.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.175+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a227400 2024-09-13T23:43:52.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.179+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94314429807360] v35'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.179+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.179+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94314429808512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9400 con 0x55c74a227400 2024-09-13T23:43:52.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.179+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9800 con 0x55c74a227c00 2024-09-13T23:43:52.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.183+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94314429808512] v35'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.183+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.183+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94314429809664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9000 con 0x55c74a227c00 2024-09-13T23:43:52.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.183+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9c00 con 0x55c74a2b8000 2024-09-13T23:43:52.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.187+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.187+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94314429810816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a226800 con 0x55c74a2b8000 2024-09-13T23:43:52.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.187+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a227800 con 0x55c74a227c00 2024-09-13T23:43:52.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.187+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94314429809664] v35'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.187+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94314429810816] v35'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.191+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.191+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94314429811968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a226800 con 0x55c74a227c00 2024-09-13T23:43:52.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.191+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8800 2024-09-13T23:43:52.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.191+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.191+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94314429813120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9400 con 0x55c74a2b8800 2024-09-13T23:43:52.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.191+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9c00 con 0x55c74a2b8800 2024-09-13T23:43:52.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.191+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94314429811968] v35'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.191+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94314429813120] v35'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94314429814272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b8400 con 0x55c74a2b8800 2024-09-13T23:43:52.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9400 con 0x55c74a2b8000 2024-09-13T23:43:52.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'171 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94314429815424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9c00 con 0x55c74a2b8000 2024-09-13T23:43:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7e9d4f40 20 add_watcher() i=0 2024-09-13T23:43:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7e9d4f40 20 add_watcher() i=1 2024-09-13T23:43:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7e9d4f40 20 add_watcher() i=2 2024-09-13T23:43:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7e9d4f40 20 add_watcher() i=3 2024-09-13T23:43:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7e9d4f40 20 add_watcher() i=4 2024-09-13T23:43:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7e9d4f40 20 add_watcher() i=5 2024-09-13T23:43:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94314429814272] v35'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.195+0000 7fda7e9d4f40 20 add_watcher() i=6 2024-09-13T23:43:52.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.199+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94314429815424] v35'172 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.199+0000 7fda7e9d4f40 20 add_watcher() i=7 2024-09-13T23:43:52.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.199+0000 7fda7e9d4f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:52.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.199+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2c8000 con 0x55c74a227400 2024-09-13T23:43:52.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.199+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.199+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94314431299584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2c8400 con 0x55c74a227400 2024-09-13T23:43:52.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.199+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94314431299584] v35'178 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.199+0000 7fda7e9d4f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:52.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.199+0000 7fda7e9d4f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:52.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.199+0000 7fda7e9d4f40 1 Processor -- start 2024-09-13T23:43:52.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.199+0000 7fda6415b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:52.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 -- start start 2024-09-13T23:43:52.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 0x55c74a16b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749f103c0 con 0x55c74a2c8c00 2024-09-13T23:43:52.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7c98c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 0x55c74a16b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7c98c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 0x55c74a16b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52364/0 (socket says 172.21.15.167:52364) 2024-09-13T23:43:52.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7c98c700 1 -- 172.21.15.167:0/1601723029 learned_addr learned my addr 172.21.15.167:0/1601723029 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7c98c700 1 -- 172.21.15.167:0/1601723029 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c749e94b60 con 0x55c74a2c8c00 2024-09-13T23:43:52.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7c98c700 1 --2- 172.21.15.167:0/1601723029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 0x55c74a16b180 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55c74a2d4030 tx=0x55c749f87860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba131deb920551d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda6395a700 1 -- 172.21.15.167:0/1601723029 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c749f27760 con 0x55c74a2c8c00 2024-09-13T23:43:52.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda6395a700 1 -- 172.21.15.167:0/1601723029 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c74a246900 con 0x55c74a2c8c00 2024-09-13T23:43:52.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda6395a700 1 -- 172.21.15.167:0/1601723029 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c74a2ca160 con 0x55c74a2c8c00 2024-09-13T23:43:52.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1601723029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 msgr2=0x55c74a16b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/1601723029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 0x55c74a16b180 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x55c74a2d4030 tx=0x55c749f87860 comp rx=0 tx=0).stop 2024-09-13T23:43:52.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1601723029 shutdown_connections 2024-09-13T23:43:52.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/1601723029 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 0x55c74a16b180 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1601723029 >> 172.21.15.167:0/1601723029 conn(0x55c74a2c8800 msgr2=0x55c749f84000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1601723029 shutdown_connections 2024-09-13T23:43:52.214 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1601723029 wait complete. 2024-09-13T23:43:52.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 Processor -- start 2024-09-13T23:43:52.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 -- start start 2024-09-13T23:43:52.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 0x55c74a16b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.203+0000 7fda7e9d4f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c749f10500 con 0x55c74a2c8c00 2024-09-13T23:43:52.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda7c98c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 0x55c74a16b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda7c98c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 0x55c74a16b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52372/0 (socket says 172.21.15.167:52372) 2024-09-13T23:43:52.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda7c98c700 1 -- 172.21.15.167:0/4044869000 learned_addr learned my addr 172.21.15.167:0/4044869000 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda7c98c700 1 -- 172.21.15.167:0/4044869000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c749e94d00 con 0x55c74a2c8c00 2024-09-13T23:43:52.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda7c98c700 1 --2- 172.21.15.167:0/4044869000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 0x55c74a16b180 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55c74a2d4240 tx=0x55c74a298420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda62157700 1 -- 172.21.15.167:0/4044869000 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c74a2ca2c0 con 0x55c74a2c8c00 2024-09-13T23:43:52.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda62157700 1 -- 172.21.15.167:0/4044869000 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c74a246600 con 0x55c74a2c8c00 2024-09-13T23:43:52.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/4044869000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c749e94340 con 0x55c74a2c8c00 2024-09-13T23:43:52.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda62157700 1 -- 172.21.15.167:0/4044869000 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c74a2ca420 con 0x55c74a2c8c00 2024-09-13T23:43:52.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/4044869000 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c749e95d40 con 0x55c74a2c8c00 2024-09-13T23:43:52.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda62157700 1 -- 172.21.15.167:0/4044869000 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c749f7d8c0 con 0x55c74a2c8c00 2024-09-13T23:43:52.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda62157700 1 --2- 172.21.15.167:0/4044869000 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a350000 0x55c74a356000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda62157700 1 -- 172.21.15.167:0/4044869000 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55c749ee96c0 con 0x55c74a2c8c00 2024-09-13T23:43:52.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda7d98e700 1 --2- 172.21.15.167:0/4044869000 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a350000 0x55c74a356000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.207+0000 7fda7e9d4f40 5 note: GC not initialized 2024-09-13T23:43:52.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7d98e700 1 --2- 172.21.15.167:0/4044869000 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a350000 0x55c74a356000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55c74a2632c0 tx=0x55c74a1ded20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda5f952700 20 reqs_thread_entry: start 2024-09-13T23:43:52.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:52.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:52.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda5d94e700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:22 2024 2024-09-13T23:43:52.220 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:52.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda5d12c700 20 reqs_thread_entry: start 2024-09-13T23:43:52.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-13T23:43:52.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a350400 con 0x55c74a2b8000 2024-09-13T23:43:52.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:52.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-13T23:43:52.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:52.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 : miss 2024-09-13T23:43:52.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.2 8:444f11e7:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a350800 con 0x55c74a2b8000 2024-09-13T23:43:52.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(63 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:52.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 info.flags=0x17 2024-09-13T23:43:52.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 to cache LRU end 2024-09-13T23:43:52.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:52.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:52.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 2024-09-13T23:43:52.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 20 get_obj_state: rctx=0x55c749f13880 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x55c74a3c01e8 s->prefetch_data=0 2024-09-13T23:43:52.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.211+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 3.1fs0 3:fb2b0a4f:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a350c00 con 0x55c74a227400 2024-09-13T23:43:52.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.215+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(64 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=847b,stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 262+0+863 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.215+0000 7fda7e9d4f40 10 manifest: total_size = 5 2024-09-13T23:43:52.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.215+0000 7fda7e9d4f40 20 get_obj_state: setting s->obj_tag to _8Q8tzMUi8S3xiUytNIbqeuWGYObp0ne 2024-09-13T23:43:52.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.215+0000 7fda6415b700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:52.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.215+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94314431299584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a351000 con 0x55c74a227400 2024-09-13T23:43:52.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.219+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94314431299584] v35'179 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.219+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94314429807360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a351400 con 0x55c74a227c00 2024-09-13T23:43:52.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.219+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94314429807360] v35'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.219+0000 7fda7e9d4f40 20 remove_watcher() i=0 2024-09-13T23:43:52.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.219+0000 7fda7e9d4f40 2 removed watcher, disabling cache 2024-09-13T23:43:52.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.219+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94314429808512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a351c00 con 0x55c74a227400 2024-09-13T23:43:52.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.223+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94314429808512] v35'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a227400 2024-09-13T23:43:52.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.223+0000 7fda7e9d4f40 20 remove_watcher() i=1 2024-09-13T23:43:52.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.223+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94314429809664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a351800 con 0x55c74a227c00 2024-09-13T23:43:52.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.223+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94314429809664] v35'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.223+0000 7fda7e9d4f40 20 remove_watcher() i=2 2024-09-13T23:43:52.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.223+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94314429810816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a350400 con 0x55c74a2b8000 2024-09-13T23:43:52.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.227+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94314429810816] v35'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.227+0000 7fda7e9d4f40 20 remove_watcher() i=3 2024-09-13T23:43:52.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.227+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94314429811968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2c8000 con 0x55c74a227c00 2024-09-13T23:43:52.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.231+0000 7fda7d18d700 1 -- 172.21.15.167:0/1381252636 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94314429811968] v35'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1200 con 0x55c74a227c00 2024-09-13T23:43:52.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.231+0000 7fda7e9d4f40 20 remove_watcher() i=4 2024-09-13T23:43:52.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.231+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94314429813120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2c8400 con 0x55c74a2b8800 2024-09-13T23:43:52.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.231+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94314429813120] v35'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.231+0000 7fda7e9d4f40 20 remove_watcher() i=5 2024-09-13T23:43:52.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.231+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94314429814272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9400 con 0x55c74a2b8800 2024-09-13T23:43:52.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94314429814272] v35'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b1440 con 0x55c74a2b8800 2024-09-13T23:43:52.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 20 remove_watcher() i=6 2024-09-13T23:43:52.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94314429815424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c74a2b9c00 con 0x55c74a2b8000 2024-09-13T23:43:52.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7d98e700 1 -- 172.21.15.167:0/1381252636 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94314429815424] v35'173 uv171 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55c74a2b0b40 con 0x55c74a2b8000 2024-09-13T23:43:52.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 20 remove_watcher() i=7 2024-09-13T23:43:52.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c74a227c00 msgr2=0x55c74a16a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c74a227c00 0x55c74a16a100 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c74a227400 msgr2=0x55c74a169b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c74a227400 0x55c74a169b80 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c74a2b8000 msgr2=0x55c74a16ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55c74a2b8000 0x55c74a16ac00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c74a2b8800 msgr2=0x55c74a16a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55c74a2b8800 0x55c74a16a680 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a227000 msgr2=0x55c74a169600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a227000 0x55c74a169600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55c74a163920 tx=0x55c749f87800 comp rx=0 tx=0).stop 2024-09-13T23:43:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 msgr2=0x55c74a168b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7c98c700 1 -- 172.21.15.167:0/1381252636 reap_dead start 2024-09-13T23:43:52.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.235+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 0x55c74a168b00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55c74a1a6570 tx=0x55c74a1defc0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 shutdown_connections 2024-09-13T23:43:52.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/1381252636 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83800 0x55c74a168b00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 >> 172.21.15.167:0/1381252636 conn(0x55c749e83400 msgr2=0x55c749f83800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 shutdown_connections 2024-09-13T23:43:52.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/1381252636 wait complete. 2024-09-13T23:43:52.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/4044869000 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a350000 msgr2=0x55c74a356000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/4044869000 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a350000 0x55c74a356000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x55c74a2632c0 tx=0x55c74a1ded20 comp rx=0 tx=0).stop 2024-09-13T23:43:52.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/4044869000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 msgr2=0x55c74a16b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/4044869000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 0x55c74a16b180 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55c74a2d4240 tx=0x55c74a298420 comp rx=0 tx=0).stop 2024-09-13T23:43:52.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/4044869000 shutdown_connections 2024-09-13T23:43:52.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/4044869000 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a350000 0x55c74a356000 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/4044869000 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c74a2c8c00 0x55c74a16b180 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/4044869000 >> 172.21.15.167:0/4044869000 conn(0x55c74a2c8800 msgr2=0x55c749f84000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/4044869000 shutdown_connections 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/4044869000 wait complete. 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2402376867 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a226000 msgr2=0x55c74a169080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/2402376867 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a226000 0x55c74a169080 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55c74a1a62d0 tx=0x55c74a1dede0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2402376867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 msgr2=0x55c74a168580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/2402376867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 0x55c74a168580 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x55c749fdff80 tx=0x55c749f87a40 comp rx=0 tx=0).stop 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2402376867 shutdown_connections 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/2402376867 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c74a226000 0x55c74a169080 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 --2- 172.21.15.167:0/2402376867 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c749e83000 0x55c74a168580 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2402376867 >> 172.21.15.167:0/2402376867 conn(0x55c749e82c00 msgr2=0x55c749f83000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2402376867 shutdown_connections 2024-09-13T23:43:52.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.239+0000 7fda7e9d4f40 1 -- 172.21.15.167:0/2402376867 wait complete. 2024-09-13T23:43:52.253 INFO:teuthology.orchestra.run.smithi167.stdout:fooFoofooFooFULL_CONTROL 2024-09-13T23:43:52.254 DEBUG:tasks.util.rgw: raw result: fooFoofooFooFULL_CONTROL 2024-09-13T23:43:52.257 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=417 failed=False 2024-09-13T23:43:52.266 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:52.267 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-09-13T23:43:52.267 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-09-13T23:43:52.267 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-09-13T23:43:52.306 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:52.306 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:52.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f9b33bf40 1 Processor -- start 2024-09-13T23:43:52.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f9b33bf40 1 -- start start 2024-09-13T23:43:52.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f9b33bf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55982b7a7000 0x55982ba8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f9b33bf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55982b86b500 con 0x55982b7a7400 2024-09-13T23:43:52.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f9b33bf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55982b834a00 con 0x55982b7a7000 2024-09-13T23:43:52.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f99af4700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55982b7a7000 0x55982ba8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f992f3700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55982b7a7400 0x55982b8a6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:49346/0 (socket says 172.21.15.167:49346) 2024-09-13T23:43:52.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f992f3700 1 -- 172.21.15.167:0/717367843 learned_addr learned my addr 172.21.15.167:0/717367843 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:52.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f99af4700 1 -- 172.21.15.167:0/717367843 >> v1:172.21.15.12:6789/0 conn(0x55982b7a7400 legacy=0x55982b8a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f99af4700 1 -- 172.21.15.167:0/717367843 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55982b7b9d40 con 0x55982b7a7000 2024-09-13T23:43:52.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f99af4700 1 --2- 172.21.15.167:0/717367843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7000 0x55982ba8c580 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x55982ba87a70 tx=0x55982b8ab920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22243838673d8e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f98af2700 1 -- 172.21.15.167:0/717367843 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982b84b4a0 con 0x55982b7a7000 2024-09-13T23:43:52.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f98af2700 1 -- 172.21.15.167:0/717367843 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55982b86b800 con 0x55982b7a7000 2024-09-13T23:43:52.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f98af2700 1 -- 172.21.15.167:0/717367843 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982b84b600 con 0x55982b7a7000 2024-09-13T23:43:52.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/717367843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7000 msgr2=0x55982ba8c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.299+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/717367843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7000 0x55982ba8c580 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x55982ba87a70 tx=0x55982b8ab920 comp rx=0 tx=0).stop 2024-09-13T23:43:52.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/717367843 shutdown_connections 2024-09-13T23:43:52.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/717367843 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7000 0x55982ba8c580 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/717367843 >> 172.21.15.167:0/717367843 conn(0x55982b7a6c00 msgr2=0x55982b8a6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/717367843 shutdown_connections 2024-09-13T23:43:52.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/717367843 wait complete. 2024-09-13T23:43:52.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 Processor -- start 2024-09-13T23:43:52.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 -- start start 2024-09-13T23:43:52.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7000 0x55982ba8c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982b834500 con 0x55982b7a7000 2024-09-13T23:43:52.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f99af4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7000 0x55982ba8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f99af4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7000 0x55982ba8c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52386/0 (socket says 172.21.15.167:52386) 2024-09-13T23:43:52.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f99af4700 1 -- 172.21.15.167:0/3989814730 learned_addr learned my addr 172.21.15.167:0/3989814730 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f99af4700 1 -- 172.21.15.167:0/3989814730 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55982b7b9ba0 con 0x55982b7a7000 2024-09-13T23:43:52.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f99af4700 1 --2- 172.21.15.167:0/3989814730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7000 0x55982ba8c580 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x55982c60be00 tx=0x55982b8ab800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1436c829e1e6f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f982f1700 1 -- 172.21.15.167:0/3989814730 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982b84b760 con 0x55982b7a7000 2024-09-13T23:43:52.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f982f1700 1 -- 172.21.15.167:0/3989814730 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55982b86bb00 con 0x55982b7a7000 2024-09-13T23:43:52.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f982f1700 1 -- 172.21.15.167:0/3989814730 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982b84b4a0 con 0x55982b7a7000 2024-09-13T23:43:52.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/3989814730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7000 msgr2=0x55982ba8c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/3989814730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7000 0x55982ba8c580 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x55982c60be00 tx=0x55982b8ab800 comp rx=0 tx=0).stop 2024-09-13T23:43:52.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/3989814730 shutdown_connections 2024-09-13T23:43:52.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/3989814730 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7000 0x55982ba8c580 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/3989814730 >> 172.21.15.167:0/3989814730 conn(0x55982b7a6c00 msgr2=0x55982b8a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/3989814730 shutdown_connections 2024-09-13T23:43:52.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/3989814730 wait complete. 2024-09-13T23:43:52.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.303+0000 7f7f9b33bf40 1 Processor -- start 2024-09-13T23:43:52.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f9b33bf40 1 -- start start 2024-09-13T23:43:52.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f9b33bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7400 0x55982ba8cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f9b33bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982b8343c0 con 0x55982b7a7400 2024-09-13T23:43:52.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f99af4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7400 0x55982ba8cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f99af4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7400 0x55982ba8cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52388/0 (socket says 172.21.15.167:52388) 2024-09-13T23:43:52.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f99af4700 1 -- 172.21.15.167:0/581758157 learned_addr learned my addr 172.21.15.167:0/581758157 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.316 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f99af4700 1 -- 172.21.15.167:0/581758157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55982b7b9a00 con 0x55982b7a7400 2024-09-13T23:43:52.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f99af4700 1 --2- 172.21.15.167:0/581758157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7400 0x55982ba8cb00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55982c60bf80 tx=0x55982b8ab920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f96aee700 1 -- 172.21.15.167:0/581758157 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982b84b600 con 0x55982b7a7400 2024-09-13T23:43:52.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f96aee700 1 -- 172.21.15.167:0/581758157 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55982b86bc80 con 0x55982b7a7400 2024-09-13T23:43:52.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/581758157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55982b7b9860 con 0x55982b7a7400 2024-09-13T23:43:52.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f96aee700 1 -- 172.21.15.167:0/581758157 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982b84b8c0 con 0x55982b7a7400 2024-09-13T23:43:52.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/581758157 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55982b7b96c0 con 0x55982b7a7400 2024-09-13T23:43:52.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f9b33bf40 1 Processor -- start 2024-09-13T23:43:52.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f9b33bf40 1 -- start start 2024-09-13T23:43:52.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f9b33bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 0x55982ba8d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f9b33bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982b7cc640 con 0x55982b7a7c00 2024-09-13T23:43:52.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.307+0000 7f7f96aee700 1 -- 172.21.15.167:0/581758157 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55982b8a0b00 con 0x55982b7a7400 2024-09-13T23:43:52.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9a2f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 0x55982ba8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9a2f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 0x55982ba8d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52394/0 (socket says 172.21.15.167:52394) 2024-09-13T23:43:52.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/2894625814 learned_addr learned my addr 172.21.15.167:0/2894625814 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f96aee700 1 --2- 172.21.15.167:0/581758157 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bb4c000 0x55982ba8e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f96aee700 1 -- 172.21.15.167:0/581758157 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55982b80d180 con 0x55982b7a7400 2024-09-13T23:43:52.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f99af4700 1 --2- 172.21.15.167:0/581758157 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bb4c000 0x55982ba8e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/2894625814 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55982b7b8000 con 0x55982b7a7c00 2024-09-13T23:43:52.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9a2f5700 1 --2- 172.21.15.167:0/2894625814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 0x55982ba8d080 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55982bb264b0 tx=0x55982bb0eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7e302ce522480b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f99af4700 1 --2- 172.21.15.167:0/581758157 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bb4c000 0x55982ba8e100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55982bacb500 tx=0x55982bb0ec60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f942e9700 1 -- 172.21.15.167:0/2894625814 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982b84b600 con 0x55982b7a7c00 2024-09-13T23:43:52.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f942e9700 1 -- 172.21.15.167:0/2894625814 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55982bb52000 con 0x55982b7a7c00 2024-09-13T23:43:52.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f942e9700 1 -- 172.21.15.167:0/2894625814 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982b84b8c0 con 0x55982b7a7c00 2024-09-13T23:43:52.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/2894625814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 msgr2=0x55982ba8d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/2894625814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 0x55982ba8d080 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55982bb264b0 tx=0x55982bb0eb40 comp rx=0 tx=0).stop 2024-09-13T23:43:52.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/2894625814 shutdown_connections 2024-09-13T23:43:52.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/2894625814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 0x55982ba8d080 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/2894625814 >> 172.21.15.167:0/2894625814 conn(0x55982b7a7800 msgr2=0x55982b8a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/2894625814 shutdown_connections 2024-09-13T23:43:52.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/2894625814 wait complete. 2024-09-13T23:43:52.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9b33bf40 1 Processor -- start 2024-09-13T23:43:52.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9b33bf40 1 -- start start 2024-09-13T23:43:52.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9b33bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 0x55982ba8d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.311+0000 7f7f9b33bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982b7cc8c0 con 0x55982b7a7c00 2024-09-13T23:43:52.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f9a2f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 0x55982ba8d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f9a2f5700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 0x55982ba8d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52396/0 (socket says 172.21.15.167:52396) 2024-09-13T23:43:52.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 learned_addr learned my addr 172.21.15.167:0/618257087 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55982b7b8ea0 con 0x55982b7a7c00 2024-09-13T23:43:52.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f9a2f5700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 0x55982ba8d080 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55982baca210 tx=0x55982bb0e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f92ae6700 1 -- 172.21.15.167:0/618257087 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982b84b760 con 0x55982b7a7c00 2024-09-13T23:43:52.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f92ae6700 1 -- 172.21.15.167:0/618257087 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55982bb52180 con 0x55982b7a7c00 2024-09-13T23:43:52.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55982b7b8820 con 0x55982b7a7c00 2024-09-13T23:43:52.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f92ae6700 1 -- 172.21.15.167:0/618257087 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982b84b600 con 0x55982b7a7c00 2024-09-13T23:43:52.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55982b7b89c0 con 0x55982b7a7c00 2024-09-13T23:43:52.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f92ae6700 1 -- 172.21.15.167:0/618257087 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55982b8a0dc0 con 0x55982b7a7c00 2024-09-13T23:43:52.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f92ae6700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bb4cc00 0x55982ba8e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.315+0000 7f7f92ae6700 1 -- 172.21.15.167:0/618257087 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55982b80d340 con 0x55982b7a7c00 2024-09-13T23:43:52.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f992f3700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bb4cc00 0x55982ba8e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55982bb4d000 0x55982ba8d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bb4d400 con 0x55982bb4d000 2024-09-13T23:43:52.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9a2f5700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55982bb4d000 0x55982ba8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f992f3700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bb4cc00 0x55982ba8e680 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55982bb27d10 tx=0x55982bbf00c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9a2f5700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55982bb4d000 0x55982ba8d600 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 20 realm 2024-09-13T23:43:52.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bb4d800 con 0x55982bb4d000 2024-09-13T23:43:52.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:52.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bb4dc00 con 0x55982bb4d000 2024-09-13T23:43:52.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55982bb4d400 0x55982ba8db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bb4d800 con 0x55982bb4d400 2024-09-13T23:43:52.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f99af4700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55982bb4d400 0x55982ba8db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.319+0000 7f7f99af4700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55982bb4d400 0x55982ba8db80 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f9b33bf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:52.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bb4dc00 con 0x55982bb4d400 2024-09-13T23:43:52.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f9b33bf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:52.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f9b33bf40 20 searching for the correct realm 2024-09-13T23:43:52.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55982bb4d800 0x55982ba8ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bb4dc00 con 0x55982bb4d800 2024-09-13T23:43:52.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f992f3700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55982bb4d800 0x55982ba8ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f992f3700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55982bb4d800 0x55982ba8ec00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bb4dc00 con 0x55982bb4d400 2024-09-13T23:43:52.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f99af4700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55982bb4dc00 0x55982ba8c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4dc00 2024-09-13T23:43:52.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.323+0000 7f7f9a2f5700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55982bb4dc00 0x55982ba8c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f9a2f5700 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55982bb4dc00 0x55982ba8c580 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d000 2024-09-13T23:43:52.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d000 2024-09-13T23:43:52.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d800 2024-09-13T23:43:52.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d000 2024-09-13T23:43:52.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d400 2024-09-13T23:43:52.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.327+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d400 2024-09-13T23:43:52.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.331+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.331+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d000 2024-09-13T23:43:52.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.331+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.331+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4dc00 2024-09-13T23:43:52.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.331+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.331+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d000 2024-09-13T23:43:52.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.331+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.331+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4dc00 2024-09-13T23:43:52.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.331+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.331+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d400 2024-09-13T23:43:52.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.331+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.331+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d800 2024-09-13T23:43:52.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d000 2024-09-13T23:43:52.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d400 2024-09-13T23:43:52.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d800 2024-09-13T23:43:52.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d000 2024-09-13T23:43:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4dc00 2024-09-13T23:43:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d000 2024-09-13T23:43:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.335+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d800 2024-09-13T23:43:52.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.339+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.339+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4dc00 2024-09-13T23:43:52.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.339+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.339+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d000 2024-09-13T23:43:52.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.339+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.339+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4dc00 2024-09-13T23:43:52.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.339+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.339+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d800 2024-09-13T23:43:52.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.339+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.339+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4dc00 2024-09-13T23:43:52.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.339+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.339+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d400 2024-09-13T23:43:52.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d400 2024-09-13T23:43:52.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d800 2024-09-13T23:43:52.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d400 2024-09-13T23:43:52.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.352 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d400 2024-09-13T23:43:52.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f9b33bf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:52.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f9b33bf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:52.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f9b33bf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:52.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f9b33bf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:52.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18800 con 0x55982bb4d000 2024-09-13T23:43:52.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f9b33bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.354 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.343+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18c00 con 0x55982bb4d400 2024-09-13T23:43:52.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:52.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19000 con 0x55982bb4d400 2024-09-13T23:43:52.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:52.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 zone default found 2024-09-13T23:43:52.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 4 Realm: () 2024-09-13T23:43:52.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:52.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:52.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:52.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 zonegroup default 2024-09-13T23:43:52.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19400 con 0x55982bb4dc00 2024-09-13T23:43:52.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19800 con 0x55982bb4d800 2024-09-13T23:43:52.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19c00 con 0x55982bb4d800 2024-09-13T23:43:52.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 started sync module instance, tier type = 2024-09-13T23:43:52.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:52.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.347+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19800 con 0x55982bb4d400 2024-09-13T23:43:52.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.351+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.351+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94112055835392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19c00 con 0x55982bb4d400 2024-09-13T23:43:52.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.351+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18400 con 0x55982bb4d000 2024-09-13T23:43:52.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.355+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.355+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94112055836544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d000 2024-09-13T23:43:52.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.355+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18c00 con 0x55982bb4d400 2024-09-13T23:43:52.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.355+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94112055835392] v35'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.366 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.355+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94112055836544] v35'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.359+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.359+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94112055837696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19000 con 0x55982bb4d400 2024-09-13T23:43:52.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.359+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18800 con 0x55982bb4dc00 2024-09-13T23:43:52.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.359+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.359+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94112055838848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19400 con 0x55982bb4dc00 2024-09-13T23:43:52.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.359+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18400 con 0x55982bb4d400 2024-09-13T23:43:52.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.359+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94112055837696] v35'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.363+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94112055838848] v35'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.363+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.363+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94112055840000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18000 con 0x55982bb4d400 2024-09-13T23:43:52.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.363+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19800 con 0x55982bb4d800 2024-09-13T23:43:52.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.363+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.363+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94112055840000] v35'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.363+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94112055841152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19c00 con 0x55982bb4d800 2024-09-13T23:43:52.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.363+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18c00 con 0x55982bb4d800 2024-09-13T23:43:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.367+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94112055841152] v35'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.367+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.367+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94112055842304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19000 con 0x55982bb4d800 2024-09-13T23:43:52.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.367+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18400 con 0x55982bb4dc00 2024-09-13T23:43:52.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94112055842304] v35'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'174 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94112055843456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19800 con 0x55982bb4dc00 2024-09-13T23:43:52.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9b33bf40 20 add_watcher() i=0 2024-09-13T23:43:52.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9b33bf40 20 add_watcher() i=1 2024-09-13T23:43:52.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9b33bf40 20 add_watcher() i=2 2024-09-13T23:43:52.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9b33bf40 20 add_watcher() i=3 2024-09-13T23:43:52.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9b33bf40 20 add_watcher() i=4 2024-09-13T23:43:52.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9b33bf40 20 add_watcher() i=5 2024-09-13T23:43:52.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9b33bf40 20 add_watcher() i=6 2024-09-13T23:43:52.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94112055843456] v35'175 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9b33bf40 20 add_watcher() i=7 2024-09-13T23:43:52.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9b33bf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:52.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc19c00 con 0x55982bb4d000 2024-09-13T23:43:52.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.371+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94112057516032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18c00 con 0x55982bb4d000 2024-09-13T23:43:52.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.375+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94112057516032] v35'180 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.375+0000 7f7f80ac2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:52.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.375+0000 7f7f9b33bf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:52.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.375+0000 7f7f9b33bf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:52.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.375+0000 7f7f9b33bf40 1 Processor -- start 2024-09-13T23:43:52.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.375+0000 7f7f9b33bf40 1 -- start start 2024-09-13T23:43:52.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.375+0000 7f7f9b33bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 0x55982ba8f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.375+0000 7f7f9b33bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982b8343c0 con 0x55982bc19400 2024-09-13T23:43:52.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.375+0000 7f7f992f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 0x55982ba8f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.375+0000 7f7f992f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 0x55982ba8f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52408/0 (socket says 172.21.15.167:52408) 2024-09-13T23:43:52.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.375+0000 7f7f992f3700 1 -- 172.21.15.167:0/1382917366 learned_addr learned my addr 172.21.15.167:0/1382917366 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.375+0000 7f7f992f3700 1 -- 172.21.15.167:0/1382917366 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55982b7b8b60 con 0x55982bc19400 2024-09-13T23:43:52.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f992f3700 1 --2- 172.21.15.167:0/1382917366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 0x55982ba8f180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x55982bc26090 tx=0x55982bbf0060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e4d2d28e0803f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f802c1700 1 -- 172.21.15.167:0/1382917366 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982bc2e000 con 0x55982bc19400 2024-09-13T23:43:52.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f802c1700 1 -- 172.21.15.167:0/1382917366 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55982bb52600 con 0x55982bc19400 2024-09-13T23:43:52.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f802c1700 1 -- 172.21.15.167:0/1382917366 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982bc2e160 con 0x55982bc19400 2024-09-13T23:43:52.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1382917366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 msgr2=0x55982ba8f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/1382917366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 0x55982ba8f180 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x55982bc26090 tx=0x55982bbf0060 comp rx=0 tx=0).stop 2024-09-13T23:43:52.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1382917366 shutdown_connections 2024-09-13T23:43:52.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/1382917366 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 0x55982ba8f180 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1382917366 >> 172.21.15.167:0/1382917366 conn(0x55982bc18800 msgr2=0x55982b8a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1382917366 shutdown_connections 2024-09-13T23:43:52.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1382917366 wait complete. 2024-09-13T23:43:52.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f9b33bf40 1 Processor -- start 2024-09-13T23:43:52.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f9b33bf40 1 -- start start 2024-09-13T23:43:52.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f9b33bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 0x55982ba8f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f9b33bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982b834500 con 0x55982bc19400 2024-09-13T23:43:52.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f992f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 0x55982ba8f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f992f3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 0x55982ba8f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52420/0 (socket says 172.21.15.167:52420) 2024-09-13T23:43:52.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.379+0000 7f7f992f3700 1 -- 172.21.15.167:0/1323492965 learned_addr learned my addr 172.21.15.167:0/1323492965 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f992f3700 1 -- 172.21.15.167:0/1323492965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55982b7b8d00 con 0x55982bc19400 2024-09-13T23:43:52.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f992f3700 1 --2- 172.21.15.167:0/1323492965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 0x55982ba8f180 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x55982bc27350 tx=0x55982bbf0180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f7eabe700 1 -- 172.21.15.167:0/1323492965 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982bc2e2c0 con 0x55982bc19400 2024-09-13T23:43:52.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f7eabe700 1 -- 172.21.15.167:0/1323492965 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55982bb52300 con 0x55982bc19400 2024-09-13T23:43:52.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1323492965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55982b7b8340 con 0x55982bc19400 2024-09-13T23:43:52.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f7eabe700 1 -- 172.21.15.167:0/1323492965 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55982bc2e000 con 0x55982bc19400 2024-09-13T23:43:52.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1323492965 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55982b7b8b60 con 0x55982bc19400 2024-09-13T23:43:52.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f7eabe700 1 -- 172.21.15.167:0/1323492965 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55982b8a1b80 con 0x55982bc19400 2024-09-13T23:43:52.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f7eabe700 1 --2- 172.21.15.167:0/1323492965 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bcaa000 0x55982bcb0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f7eabe700 1 -- 172.21.15.167:0/1323492965 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55982b80d500 con 0x55982bc19400 2024-09-13T23:43:52.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f9b33bf40 5 note: GC not initialized 2024-09-13T23:43:52.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f9a2f5700 1 --2- 172.21.15.167:0/1323492965 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bcaa000 0x55982bcb0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.383+0000 7f7f9a2f5700 1 --2- 172.21.15.167:0/1323492965 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bcaa000 0x55982bcb0000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55982bba9590 tx=0x55982bb0f500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f7c2b9700 20 reqs_thread_entry: start 2024-09-13T23:43:52.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:52.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:52.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f7a2b5700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:22 2024 2024-09-13T23:43:52.396 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:52.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f79a93700 20 reqs_thread_entry: start 2024-09-13T23:43:52.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-13T23:43:52.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bcaa800 con 0x55982bb4dc00 2024-09-13T23:43:52.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:52.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-13T23:43:52.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:52.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 : miss 2024-09-13T23:43:52.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.2 8:444f11e7:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bcaac00 con 0x55982bb4dc00 2024-09-13T23:43:52.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(63 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:52.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 info.flags=0x17 2024-09-13T23:43:52.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 to cache LRU end 2024-09-13T23:43:52.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:52.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:52.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 2024-09-13T23:43:52.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 20 get_obj_state: rctx=0x55982b8378f0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x55982bd121e8 s->prefetch_data=0 2024-09-13T23:43:52.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 3.1fs0 3:fb2b0a4f:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bcab000 con 0x55982bb4d000 2024-09-13T23:43:52.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(64 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=909b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+925 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 10 manifest: total_size = 5 2024-09-13T23:43:52.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.387+0000 7f7f9b33bf40 20 get_obj_state: setting s->obj_tag to _CtZaybV8LtsgmEBRAh2kNBN-wLlFOcF 2024-09-13T23:43:52.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.391+0000 7f7f80ac2700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:52.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.391+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94112057516032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bcab400 con 0x55982bb4d000 2024-09-13T23:43:52.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.395+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94112057516032] v35'181 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.395+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94112055835392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bcab800 con 0x55982bb4d400 2024-09-13T23:43:52.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.395+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94112055835392] v35'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.395+0000 7f7f9b33bf40 20 remove_watcher() i=0 2024-09-13T23:43:52.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.395+0000 7f7f9b33bf40 2 removed watcher, disabling cache 2024-09-13T23:43:52.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.395+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94112055836544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bcabc00 con 0x55982bb4d000 2024-09-13T23:43:52.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.399+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94112055836544] v35'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4d000 2024-09-13T23:43:52.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.399+0000 7f7f9b33bf40 20 remove_watcher() i=1 2024-09-13T23:43:52.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.399+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94112055837696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bcaa800 con 0x55982bb4d400 2024-09-13T23:43:52.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.399+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94112055837696] v35'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.399+0000 7f7f9b33bf40 20 remove_watcher() i=2 2024-09-13T23:43:52.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.399+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94112055838848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bcaac00 con 0x55982bb4dc00 2024-09-13T23:43:52.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.403+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94112055838848] v35'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.403+0000 7f7f9b33bf40 20 remove_watcher() i=3 2024-09-13T23:43:52.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.403+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94112055840000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bcab000 con 0x55982bb4d400 2024-09-13T23:43:52.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.407+0000 7f7f99af4700 1 -- 172.21.15.167:0/618257087 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94112055840000] v35'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d200 con 0x55982bb4d400 2024-09-13T23:43:52.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.407+0000 7f7f9b33bf40 20 remove_watcher() i=4 2024-09-13T23:43:52.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.407+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94112055841152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bcab400 con 0x55982bb4d800 2024-09-13T23:43:52.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.407+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94112055841152] v35'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.407+0000 7f7f9b33bf40 20 remove_watcher() i=5 2024-09-13T23:43:52.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.407+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94112055842304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bcabc00 con 0x55982bb4d800 2024-09-13T23:43:52.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.411+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94112055842304] v35'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9d440 con 0x55982bb4d800 2024-09-13T23:43:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.411+0000 7f7f9b33bf40 20 remove_watcher() i=6 2024-09-13T23:43:52.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.411+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94112055843456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55982bc18c00 con 0x55982bb4dc00 2024-09-13T23:43:52.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9a2f5700 1 -- 172.21.15.167:0/618257087 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94112055843456] v35'176 uv174 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55982bb9cb40 con 0x55982bb4dc00 2024-09-13T23:43:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 20 remove_watcher() i=7 2024-09-13T23:43:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55982bb4d400 msgr2=0x55982ba8db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55982bb4d400 0x55982ba8db80 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55982bb4d000 msgr2=0x55982ba8d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55982bb4d000 0x55982ba8d600 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55982bb4dc00 msgr2=0x55982ba8c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55982bb4dc00 0x55982ba8c580 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55982bb4d800 msgr2=0x55982ba8ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55982bb4d800 0x55982ba8ec00 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bb4cc00 msgr2=0x55982ba8e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bb4cc00 0x55982ba8e680 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x55982bb27d10 tx=0x55982bbf00c0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 msgr2=0x55982ba8d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 0x55982ba8d080 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x55982baca210 tx=0x55982bb0e5a0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f992f3700 1 -- 172.21.15.167:0/618257087 reap_dead start 2024-09-13T23:43:52.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 shutdown_connections 2024-09-13T23:43:52.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/618257087 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7c00 0x55982ba8d080 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 >> 172.21.15.167:0/618257087 conn(0x55982b7a7800 msgr2=0x55982b8a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 shutdown_connections 2024-09-13T23:43:52.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/618257087 wait complete. 2024-09-13T23:43:52.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1323492965 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bcaa000 msgr2=0x55982bcb0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/1323492965 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bcaa000 0x55982bcb0000 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x55982bba9590 tx=0x55982bb0f500 comp rx=0 tx=0).stop 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1323492965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 msgr2=0x55982ba8f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/1323492965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 0x55982ba8f180 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x55982bc27350 tx=0x55982bbf0180 comp rx=0 tx=0).stop 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1323492965 shutdown_connections 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/1323492965 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bcaa000 0x55982bcb0000 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/1323492965 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982bc19400 0x55982ba8f180 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1323492965 >> 172.21.15.167:0/1323492965 conn(0x55982bc18800 msgr2=0x55982b8a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1323492965 shutdown_connections 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/1323492965 wait complete. 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/581758157 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bb4c000 msgr2=0x55982ba8e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/581758157 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bb4c000 0x55982ba8e100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55982bacb500 tx=0x55982bb0ec60 comp rx=0 tx=0).stop 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/581758157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7400 msgr2=0x55982ba8cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/581758157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7400 0x55982ba8cb00 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55982c60bf80 tx=0x55982b8ab920 comp rx=0 tx=0).stop 2024-09-13T23:43:52.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/581758157 shutdown_connections 2024-09-13T23:43:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/581758157 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55982bb4c000 0x55982ba8e100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 --2- 172.21.15.167:0/581758157 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55982b7a7400 0x55982ba8cb00 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/581758157 >> 172.21.15.167:0/581758157 conn(0x55982b7a6c00 msgr2=0x55982b8a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/581758157 shutdown_connections 2024-09-13T23:43:52.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.415+0000 7f7f9b33bf40 1 -- 172.21.15.167:0/581758157 wait complete. 2024-09-13T23:43:52.432 INFO:teuthology.orchestra.run.smithi167.stdout:fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-09-13T23:43:52.432 DEBUG:tasks.util.rgw: raw result: fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-09-13T23:43:52.435 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=609 failed=False 2024-09-13T23:43:52.440 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:52.518 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-09-13T23:43:52.518 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-09-13T23:43:52.518 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-09-13T23:43:52.519 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-09-13T23:43:52.519 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-09-13T23:43:52.519 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-09-13T23:43:52.519 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket rm --bucket myfoo --purge-objects 2024-09-13T23:43:52.557 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:52.557 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:52.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.547+0000 7f20aa3e0f40 1 Processor -- start 2024-09-13T23:43:52.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.547+0000 7f20aa3e0f40 1 -- start start 2024-09-13T23:43:52.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5627b9211000 0x5627b94f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5627b92d5500 con 0x5627b9211400 2024-09-13T23:43:52.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5627b929ea00 con 0x5627b9211000 2024-09-13T23:43:52.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20a8b99700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5627b9211000 0x5627b94f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20a8398700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5627b9211400 0x5627b9310800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:49354/0 (socket says 172.21.15.167:49354) 2024-09-13T23:43:52.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20a8398700 1 -- 172.21.15.167:0/3820159387 learned_addr learned my addr 172.21.15.167:0/3820159387 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:52.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20a8b99700 1 -- 172.21.15.167:0/3820159387 >> v1:172.21.15.12:6789/0 conn(0x5627b9211400 legacy=0x5627b9310800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20a8b99700 1 -- 172.21.15.167:0/3820159387 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b9223d40 con 0x5627b9211000 2024-09-13T23:43:52.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20a8b99700 1 --2- 172.21.15.167:0/3820159387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x5627b94f1a70 tx=0x5627b9315920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc4e7ebdd9bbef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20a7b97700 1 -- 172.21.15.167:0/3820159387 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b92b54a0 con 0x5627b9211000 2024-09-13T23:43:52.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20a7b97700 1 -- 172.21.15.167:0/3820159387 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b92d5800 con 0x5627b9211000 2024-09-13T23:43:52.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20a7b97700 1 -- 172.21.15.167:0/3820159387 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b92b5600 con 0x5627b9211000 2024-09-13T23:43:52.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3820159387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 msgr2=0x5627b94f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/3820159387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x5627b94f1a70 tx=0x5627b9315920 comp rx=0 tx=0).stop 2024-09-13T23:43:52.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3820159387 shutdown_connections 2024-09-13T23:43:52.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/3820159387 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3820159387 >> 172.21.15.167:0/3820159387 conn(0x5627b9210c00 msgr2=0x5627b9310000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3820159387 shutdown_connections 2024-09-13T23:43:52.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3820159387 wait complete. 2024-09-13T23:43:52.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 Processor -- start 2024-09-13T23:43:52.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 -- start start 2024-09-13T23:43:52.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.551+0000 7f20aa3e0f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b929e500 con 0x5627b9211000 2024-09-13T23:43:52.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20a8b99700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20a8b99700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52438/0 (socket says 172.21.15.167:52438) 2024-09-13T23:43:52.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20a8b99700 1 -- 172.21.15.167:0/3354111842 learned_addr learned my addr 172.21.15.167:0/3354111842 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20a8b99700 1 -- 172.21.15.167:0/3354111842 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b9223ba0 con 0x5627b9211000 2024-09-13T23:43:52.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20a8b99700 1 --2- 172.21.15.167:0/3354111842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x5627ba071800 tx=0x5627b9315800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf868651d3b9437 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20a7396700 1 -- 172.21.15.167:0/3354111842 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b92b5760 con 0x5627b9211000 2024-09-13T23:43:52.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20a7396700 1 -- 172.21.15.167:0/3354111842 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b92d5b00 con 0x5627b9211000 2024-09-13T23:43:52.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20a7396700 1 -- 172.21.15.167:0/3354111842 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b92b54a0 con 0x5627b9211000 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3354111842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 msgr2=0x5627b94f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/3354111842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x5627ba071800 tx=0x5627b9315800 comp rx=0 tx=0).stop 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3354111842 shutdown_connections 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/3354111842 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3354111842 >> 172.21.15.167:0/3354111842 conn(0x5627b9210c00 msgr2=0x5627b9311000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3354111842 shutdown_connections 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3354111842 wait complete. 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20aa3e0f40 1 Processor -- start 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20aa3e0f40 1 -- start start 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20aa3e0f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20aa3e0f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b929e3c0 con 0x5627b9211000 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20a8b99700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20a8b99700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52440/0 (socket says 172.21.15.167:52440) 2024-09-13T23:43:52.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20a8b99700 1 -- 172.21.15.167:0/3834912768 learned_addr learned my addr 172.21.15.167:0/3834912768 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.555+0000 7f20a8b99700 1 -- 172.21.15.167:0/3834912768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b9223a00 con 0x5627b9211000 2024-09-13T23:43:52.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a8b99700 1 --2- 172.21.15.167:0/3834912768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x5627ba0716b0 tx=0x5627b9315920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a5b93700 1 -- 172.21.15.167:0/3834912768 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b92b5600 con 0x5627b9211000 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a5b93700 1 -- 172.21.15.167:0/3834912768 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b92d5c80 con 0x5627b9211000 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3834912768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5627b9223860 con 0x5627b9211000 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a5b93700 1 -- 172.21.15.167:0/3834912768 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b92b58c0 con 0x5627b9211000 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3834912768 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627b92236c0 con 0x5627b9211000 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20aa3e0f40 1 Processor -- start 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20aa3e0f40 1 -- start start 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20aa3e0f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 0x5627b94f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20aa3e0f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b9236640 con 0x5627b9211800 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a5b93700 1 -- 172.21.15.167:0/3834912768 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5627b930ab00 con 0x5627b9211000 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a939a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 0x5627b94f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a5b93700 1 --2- 172.21.15.167:0/3834912768 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b95b2000 0x5627b94f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a939a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 0x5627b94f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52448/0 (socket says 172.21.15.167:52448) 2024-09-13T23:43:52.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a939a700 1 -- 172.21.15.167:0/259533661 learned_addr learned my addr 172.21.15.167:0/259533661 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a5b93700 1 -- 172.21.15.167:0/3834912768 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5627b9277180 con 0x5627b9211000 2024-09-13T23:43:52.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a8b99700 1 --2- 172.21.15.167:0/3834912768 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b95b2000 0x5627b94f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a939a700 1 -- 172.21.15.167:0/259533661 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b9222000 con 0x5627b9211800 2024-09-13T23:43:52.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a8b99700 1 --2- 172.21.15.167:0/3834912768 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b95b2000 0x5627b94f7080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5627ba0850b0 tx=0x5627b9576b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.559+0000 7f20a939a700 1 --2- 172.21.15.167:0/259533661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 0x5627b94f6b00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x5627ba084120 tx=0x5627b9576e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4deec8f5b8096b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20a338e700 1 -- 172.21.15.167:0/259533661 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b92b5600 con 0x5627b9211800 2024-09-13T23:43:52.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20a338e700 1 -- 172.21.15.167:0/259533661 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b95b8000 con 0x5627b9211800 2024-09-13T23:43:52.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20a338e700 1 -- 172.21.15.167:0/259533661 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b92b58c0 con 0x5627b9211800 2024-09-13T23:43:52.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/259533661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 msgr2=0x5627b94f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/259533661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 0x5627b94f6b00 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x5627ba084120 tx=0x5627b9576e40 comp rx=0 tx=0).stop 2024-09-13T23:43:52.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/259533661 shutdown_connections 2024-09-13T23:43:52.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/259533661 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 0x5627b94f6b00 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/259533661 >> 172.21.15.167:0/259533661 conn(0x5627b9211400 msgr2=0x5627b9311800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/259533661 shutdown_connections 2024-09-13T23:43:52.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/259533661 wait complete. 2024-09-13T23:43:52.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20aa3e0f40 1 Processor -- start 2024-09-13T23:43:52.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20aa3e0f40 1 -- start start 2024-09-13T23:43:52.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20aa3e0f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 0x5627b94f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20aa3e0f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b92368c0 con 0x5627b9211800 2024-09-13T23:43:52.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20a939a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 0x5627b94f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20a939a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 0x5627b94f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52458/0 (socket says 172.21.15.167:52458) 2024-09-13T23:43:52.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.563+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 learned_addr learned my addr 172.21.15.167:0/2161935347 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.567+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b9222ea0 con 0x5627b9211800 2024-09-13T23:43:52.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.567+0000 7f20a939a700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 0x5627b94f6b00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x5627ba084300 tx=0x5627b95765a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.567+0000 7f20a1b8b700 1 -- 172.21.15.167:0/2161935347 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b92b5760 con 0x5627b9211800 2024-09-13T23:43:52.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.567+0000 7f20a1b8b700 1 -- 172.21.15.167:0/2161935347 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b95b8180 con 0x5627b9211800 2024-09-13T23:43:52.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.567+0000 7f20a1b8b700 1 -- 172.21.15.167:0/2161935347 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b92b5600 con 0x5627b9211800 2024-09-13T23:43:52.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.567+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5627b9222820 con 0x5627b9211800 2024-09-13T23:43:52.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.567+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627b92229c0 con 0x5627b9211800 2024-09-13T23:43:52.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.567+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.567+0000 7f20a1b8b700 1 -- 172.21.15.167:0/2161935347 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5627b930b080 con 0x5627b9211800 2024-09-13T23:43:52.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20a1b8b700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b95b2c00 0x5627b94f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20a1b8b700 1 -- 172.21.15.167:0/2161935347 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5627b9277340 con 0x5627b9211800 2024-09-13T23:43:52.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20a8398700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b95b2c00 0x5627b94f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5627b95b3000 0x5627b94f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3400 con 0x5627b95b3000 2024-09-13T23:43:52.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20a939a700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5627b95b3000 0x5627b94f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20a939a700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5627b95b3000 0x5627b94f7b80 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20a8398700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b95b2c00 0x5627b94f7600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5627b95b5b30 tx=0x5627b966a0c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 20 realm 2024-09-13T23:43:52.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3000 2024-09-13T23:43:52.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:52.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3c00 con 0x5627b95b3000 2024-09-13T23:43:52.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5627b95b3400 0x5627b94f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3400 2024-09-13T23:43:52.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20a8b99700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5627b95b3400 0x5627b94f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.571+0000 7f20a8b99700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5627b95b3400 0x5627b94f8100 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20aa3e0f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:52.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3c00 con 0x5627b95b3400 2024-09-13T23:43:52.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20aa3e0f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:52.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20aa3e0f40 20 searching for the correct realm 2024-09-13T23:43:52.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5627b967a000 0x5627b94f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967a400 con 0x5627b967a000 2024-09-13T23:43:52.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20a8398700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5627b967a000 0x5627b94f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20a8398700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5627b967a000 0x5627b94f8680 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.575+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967a400 con 0x5627b95b3400 2024-09-13T23:43:52.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a8b99700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5627b967a400 0x5627b94f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a400 2024-09-13T23:43:52.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a939a700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5627b967a400 0x5627b94f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a939a700 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5627b967a400 0x5627b94f8c00 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3000 2024-09-13T23:43:52.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3000 2024-09-13T23:43:52.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a000 2024-09-13T23:43:52.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3000 2024-09-13T23:43:52.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.579+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3400 2024-09-13T23:43:52.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3400 2024-09-13T23:43:52.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3000 2024-09-13T23:43:52.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a400 2024-09-13T23:43:52.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3000 2024-09-13T23:43:52.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a400 2024-09-13T23:43:52.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3400 2024-09-13T23:43:52.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.583+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a000 2024-09-13T23:43:52.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3000 2024-09-13T23:43:52.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3400 2024-09-13T23:43:52.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a000 2024-09-13T23:43:52.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3000 2024-09-13T23:43:52.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a400 2024-09-13T23:43:52.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3000 2024-09-13T23:43:52.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.587+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a000 2024-09-13T23:43:52.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.591+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.591+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a400 2024-09-13T23:43:52.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.591+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.591+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3000 2024-09-13T23:43:52.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.591+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.591+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a400 2024-09-13T23:43:52.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.591+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.591+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a000 2024-09-13T23:43:52.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.591+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.591+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a400 2024-09-13T23:43:52.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.591+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.591+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3400 2024-09-13T23:43:52.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3400 2024-09-13T23:43:52.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a000 2024-09-13T23:43:52.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3400 2024-09-13T23:43:52.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b95b3400 2024-09-13T23:43:52.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20aa3e0f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:52.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20aa3e0f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:52.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20aa3e0f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:52.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20aa3e0f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:52.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967a800 con 0x5627b95b3000 2024-09-13T23:43:52.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20aa3e0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.595+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967ac00 con 0x5627b95b3400 2024-09-13T23:43:52.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:52.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b000 con 0x5627b95b3400 2024-09-13T23:43:52.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:52.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 zone default found 2024-09-13T23:43:52.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 4 Realm: () 2024-09-13T23:43:52.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:52.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:52.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:52.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 zonegroup default 2024-09-13T23:43:52.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b400 con 0x5627b967a400 2024-09-13T23:43:52.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b800 con 0x5627b967a000 2024-09-13T23:43:52.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967bc00 con 0x5627b967a000 2024-09-13T23:43:52.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 started sync module instance, tier type = 2024-09-13T23:43:52.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:52.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.599+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b800 con 0x5627b95b3400 2024-09-13T23:43:52.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.603+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.603+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94728612666112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967bc00 con 0x5627b95b3400 2024-09-13T23:43:52.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.603+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967ac00 con 0x5627b95b3000 2024-09-13T23:43:52.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.607+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.607+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94728612667264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b000 con 0x5627b95b3000 2024-09-13T23:43:52.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.607+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b800 con 0x5627b95b3400 2024-09-13T23:43:52.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.607+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94728612666112] v35'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.607+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94728612667264] v35'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.607+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.611+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94728612668416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3c00 con 0x5627b95b3400 2024-09-13T23:43:52.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.611+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a400 2024-09-13T23:43:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.611+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94728612668416] v35'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.611+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.611+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94728612669568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967a800 con 0x5627b967a400 2024-09-13T23:43:52.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.611+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b400 con 0x5627b95b3400 2024-09-13T23:43:52.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.615+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94728612669568] v35'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.615+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.615+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94728612670720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967ac00 con 0x5627b95b3400 2024-09-13T23:43:52.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.615+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b000 con 0x5627b967a000 2024-09-13T23:43:52.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.619+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.619+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94728612671872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680000 con 0x5627b967a000 2024-09-13T23:43:52.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.619+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94728612670720] v35'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.619+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680400 con 0x5627b967a000 2024-09-13T23:43:52.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.619+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v35'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.619+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94728612673024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680800 con 0x5627b967a000 2024-09-13T23:43:52.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.619+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680c00 con 0x5627b967a400 2024-09-13T23:43:52.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.619+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94728612671872] v35'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'177 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94728612674176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9681000 con 0x5627b967a400 2024-09-13T23:43:52.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20aa3e0f40 20 add_watcher() i=0 2024-09-13T23:43:52.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20aa3e0f40 20 add_watcher() i=1 2024-09-13T23:43:52.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20aa3e0f40 20 add_watcher() i=2 2024-09-13T23:43:52.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20aa3e0f40 20 add_watcher() i=3 2024-09-13T23:43:52.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20aa3e0f40 20 add_watcher() i=4 2024-09-13T23:43:52.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20aa3e0f40 20 add_watcher() i=5 2024-09-13T23:43:52.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94728612673024] v35'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20aa3e0f40 20 add_watcher() i=6 2024-09-13T23:43:52.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94728612674176] v35'178 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20aa3e0f40 20 add_watcher() i=7 2024-09-13T23:43:52.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20aa3e0f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:52.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9681400 con 0x5627b95b3000 2024-09-13T23:43:52.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.623+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94728614338560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9681800 con 0x5627b95b3000 2024-09-13T23:43:52.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.627+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94728614338560] v35'182 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.627+0000 7f208fb67700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:52.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.627+0000 7f20aa3e0f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:52.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.627+0000 7f20aa3e0f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:52.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.627+0000 7f20aa3e0f40 1 Processor -- start 2024-09-13T23:43:52.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.627+0000 7f20aa3e0f40 1 -- start start 2024-09-13T23:43:52.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.627+0000 7f20aa3e0f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9680c00 0x5627b94f9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.627+0000 7f20aa3e0f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b929e3c0 con 0x5627b9680c00 2024-09-13T23:43:52.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.627+0000 7f20a8398700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9680c00 0x5627b94f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.627+0000 7f20a8398700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9680c00 0x5627b94f9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52466/0 (socket says 172.21.15.167:52466) 2024-09-13T23:43:52.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.627+0000 7f20a8398700 1 -- 172.21.15.167:0/1983319946 learned_addr learned my addr 172.21.15.167:0/1983319946 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.627+0000 7f20a8398700 1 -- 172.21.15.167:0/1983319946 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b9222b60 con 0x5627b9680c00 2024-09-13T23:43:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20a8398700 1 --2- 172.21.15.167:0/1983319946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9680c00 0x5627b94f9180 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x5627b9690090 tx=0x5627b966a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0f8d8acfee62e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f208f366700 1 -- 172.21.15.167:0/1983319946 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b9698000 con 0x5627b9680c00 2024-09-13T23:43:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f208f366700 1 -- 172.21.15.167:0/1983319946 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b95b8600 con 0x5627b9680c00 2024-09-13T23:43:52.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f208f366700 1 -- 172.21.15.167:0/1983319946 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b9698160 con 0x5627b9680c00 2024-09-13T23:43:52.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/1983319946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9680c00 msgr2=0x5627b94f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/1983319946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9680c00 0x5627b94f9180 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x5627b9690090 tx=0x5627b966a060 comp rx=0 tx=0).stop 2024-09-13T23:43:52.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/1983319946 shutdown_connections 2024-09-13T23:43:52.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/1983319946 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9680c00 0x5627b94f9180 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/1983319946 >> 172.21.15.167:0/1983319946 conn(0x5627b9681c00 msgr2=0x5627b9312000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/1983319946 shutdown_connections 2024-09-13T23:43:52.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/1983319946 wait complete. 2024-09-13T23:43:52.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20aa3e0f40 1 Processor -- start 2024-09-13T23:43:52.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20aa3e0f40 1 -- start start 2024-09-13T23:43:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20aa3e0f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9681000 0x5627b94f9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20aa3e0f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627b929e500 con 0x5627b9681000 2024-09-13T23:43:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20a8398700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9681000 0x5627b94f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20a8398700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9681000 0x5627b94f9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52480/0 (socket says 172.21.15.167:52480) 2024-09-13T23:43:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20a8398700 1 -- 172.21.15.167:0/2222944702 learned_addr learned my addr 172.21.15.167:0/2222944702 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20a8398700 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5627b9222d00 con 0x5627b9681000 2024-09-13T23:43:52.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.631+0000 7f20a8398700 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9681000 0x5627b94f9700 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x5627b96902a0 tx=0x5627b966a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f208db63700 1 -- 172.21.15.167:0/2222944702 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b96982c0 con 0x5627b9681000 2024-09-13T23:43:52.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f208db63700 1 -- 172.21.15.167:0/2222944702 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5627b95b8300 con 0x5627b9681000 2024-09-13T23:43:52.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5627b9222340 con 0x5627b9681000 2024-09-13T23:43:52.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f208db63700 1 -- 172.21.15.167:0/2222944702 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5627b9698000 con 0x5627b9681000 2024-09-13T23:43:52.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5627b9222b60 con 0x5627b9681000 2024-09-13T23:43:52.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f208db63700 1 -- 172.21.15.167:0/2222944702 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5627b930bb80 con 0x5627b9681000 2024-09-13T23:43:52.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f208db63700 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b9714000 0x5627b971a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f208db63700 1 -- 172.21.15.167:0/2222944702 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5627b9277500 con 0x5627b9681000 2024-09-13T23:43:52.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5627b9681400 0x5627b971a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f20a939a700 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b9714000 0x5627b971a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680000 con 0x5627b9681400 2024-09-13T23:43:52.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f20a8b99700 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5627b9681400 0x5627b971a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f20a939a700 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b9714000 0x5627b971a000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x5627ba084f00 tx=0x5627b9577500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.635+0000 7f20a8b99700 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5627b9681400 0x5627b971a580 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.639+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v35'36 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.639+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5627b9680400 0x5627b971ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.639+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967a800 con 0x5627b9680400 2024-09-13T23:43:52.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.639+0000 7f20a8398700 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5627b9680400 0x5627b971ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.639+0000 7f20a8398700 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5627b9680400 0x5627b971ab00 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.643+0000 7f20a8398700 1 -- 172.21.15.167:0/2222944702 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v35'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b9680400 2024-09-13T23:43:52.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.643+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5627b967b000 0x5627b971b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.643+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967b000 2024-09-13T23:43:52.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.643+0000 7f20a939a700 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5627b967b000 0x5627b971b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.643+0000 7f20a939a700 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5627b967b000 0x5627b971b080 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.643+0000 7f20a939a700 1 -- 172.21.15.167:0/2222944702 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v35'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967b000 2024-09-13T23:43:52.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.643+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5627b9714400 0x5627b971b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.643+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9714800 con 0x5627b9714400 2024-09-13T23:43:52.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.643+0000 7f20a8b99700 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5627b9714400 0x5627b971b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.647+0000 7f20a8b99700 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5627b9714400 0x5627b971b600 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.647+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v35'183 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9714400 2024-09-13T23:43:52.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.647+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9714c00 con 0x5627b9681400 2024-09-13T23:43:52.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.651+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v35'62 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.651+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715000 con 0x5627b967b000 2024-09-13T23:43:52.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.655+0000 7f20a939a700 1 -- 172.21.15.167:0/2222944702 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v35'44 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967b000 2024-09-13T23:43:52.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.655+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715400 con 0x5627b9681400 2024-09-13T23:43:52.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.655+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v35'26 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.655+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715800 con 0x5627b9714400 2024-09-13T23:43:52.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.659+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v35'42 uv35 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9714400 2024-09-13T23:43:52.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.659+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715c00 con 0x5627b9681400 2024-09-13T23:43:52.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.663+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v35'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.663+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680000 con 0x5627b9681400 2024-09-13T23:43:52.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.663+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v35'31 uv28 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.663+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967bc00 con 0x5627b9681400 2024-09-13T23:43:52.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.667+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v35'63 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.667+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b800 con 0x5627b967b000 2024-09-13T23:43:52.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.671+0000 7f20a939a700 1 -- 172.21.15.167:0/2222944702 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v35'27 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967b000 2024-09-13T23:43:52.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.671+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b400 con 0x5627b9681400 2024-09-13T23:43:52.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.671+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v35'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.671+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967ac00 con 0x5627b9714400 2024-09-13T23:43:52.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.675+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v35'47 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9714400 2024-09-13T23:43:52.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.675+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3c00 con 0x5627b9714400 2024-09-13T23:43:52.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.679+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v35'184 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9714400 2024-09-13T23:43:52.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.679+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967bc00 con 0x5627b9681400 2024-09-13T23:43:52.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.683+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v35'59 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.683+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680000 con 0x5627b9681400 2024-09-13T23:43:52.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.683+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v35'64 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.683+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9714800 con 0x5627b967b000 2024-09-13T23:43:52.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.687+0000 7f20a939a700 1 -- 172.21.15.167:0/2222944702 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v35'26 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967b000 2024-09-13T23:43:52.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.687+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9714c00 con 0x5627b9681400 2024-09-13T23:43:52.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.691+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v35'32 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.691+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715400 con 0x5627b9714400 2024-09-13T23:43:52.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.695+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v35'185 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9714400 2024-09-13T23:43:52.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.695+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715800 con 0x5627b9681400 2024-09-13T23:43:52.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.695+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v35'60 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.706 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.695+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715c00 con 0x5627b9681400 2024-09-13T23:43:52.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.699+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v35'34 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.699+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9714c00 con 0x5627b967b000 2024-09-13T23:43:52.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.703+0000 7f20a939a700 1 -- 172.21.15.167:0/2222944702 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v35'26 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967b000 2024-09-13T23:43:52.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.703+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715400 con 0x5627b9681400 2024-09-13T23:43:52.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.707+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v35'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.707+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715800 con 0x5627b967b000 2024-09-13T23:43:52.719 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.707+0000 7f20a939a700 1 -- 172.21.15.167:0/2222944702 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v35'20 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967b000 2024-09-13T23:43:52.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.707+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680000 con 0x5627b967b000 2024-09-13T23:43:52.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.711+0000 7f20a939a700 1 -- 172.21.15.167:0/2222944702 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v35'45 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967b000 2024-09-13T23:43:52.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.711+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b400 con 0x5627b967b000 2024-09-13T23:43:52.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.715+0000 7f20a939a700 1 -- 172.21.15.167:0/2222944702 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v35'26 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967b000 2024-09-13T23:43:52.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.715+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967ac00 con 0x5627b9681400 2024-09-13T23:43:52.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.715+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v35'24 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.715+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967bc00 con 0x5627b9714400 2024-09-13T23:43:52.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.719+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v35'43 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9714400 2024-09-13T23:43:52.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.719+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3c00 con 0x5627b9681400 2024-09-13T23:43:52.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.723+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v35'61 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.723+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715000 con 0x5627b9714400 2024-09-13T23:43:52.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.723+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v35'48 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9714400 2024-09-13T23:43:52.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.723+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9714800 con 0x5627b9680400 2024-09-13T23:43:52.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.727+0000 7f20a8398700 1 -- 172.21.15.167:0/2222944702 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v35'30 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b9680400 2024-09-13T23:43:52.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f208b35e700 20 reqs_thread_entry: start 2024-09-13T23:43:52.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:52.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:52.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f208935a700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:23 2024 2024-09-13T23:43:52.740 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:52.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f2088b38700 20 reqs_thread_entry: start 2024-09-13T23:43:52.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-13T23:43:52.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.740 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a400 2024-09-13T23:43:52.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) v8 ==== 191+0+213 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 20 rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:52.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-09-13T23:43:52.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:52.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 : miss 2024-09-13T23:43:52.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.2 8:444f11e7:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9681800 con 0x5627b967a400 2024-09-13T23:43:52.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(63 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:43:52.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 info.flags=0x17 2024-09-13T23:43:52.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 to cache LRU end 2024-09-13T23:43:52.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:52.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:52.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 2024-09-13T23:43:52.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.731+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 4.2 4:42118f18:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982e000 con 0x5627b95b3000 2024-09-13T23:43:52.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.735+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(64 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.735+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 4.36 4:6f89048c:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982e400 con 0x5627b95b3000 2024-09-13T23:43:52.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.735+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(65 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.735+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 4.b 4:d3e0788f:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982e800 con 0x5627b967a400 2024-09-13T23:43:52.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.735+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(66 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2 [call out=425b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.735+0000 7f20aa3e0f40 10 cls_bucket_list_unordered: got 9c8454ddf7aa50116496bac348d7550d 2024-09-13T23:43:52.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.735+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 4.14 4:2b4eeb18:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982ec00 con 0x5627b967a000 2024-09-13T23:43:52.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.735+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(67 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.3 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.735+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 4.9 4:92fa94e2:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982f000 con 0x5627b95b3000 2024-09-13T23:43:52.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.735+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(68 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.735+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 4.c 4:3265cd07:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982f400 con 0x5627b95b3000 2024-09-13T23:43:52.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.739+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(69 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.5 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.739+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 4.17 4:e8760d03:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982f800 con 0x5627b967a400 2024-09-13T23:43:52.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.739+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(70 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6 [call out=425b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.739+0000 7f20aa3e0f40 10 cls_bucket_list_unordered: got b1b9a972ccd8c962a473909b97007eb4 2024-09-13T23:43:52.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.739+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:71 4.3c 4:3e7ace4d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982fc00 con 0x5627b95b3000 2024-09-13T23:43:52.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.739+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(71 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.739+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 4.0 4:0076bdb1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982ec00 con 0x5627b967a400 2024-09-13T23:43:52.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.739+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(72 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8 [call out=425b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+425 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.739+0000 7f20aa3e0f40 10 cls_bucket_list_unordered: got c785e1ed2950e3e36b1e2ca01f299a54 2024-09-13T23:43:52.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.739+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:73 4.33 4:cc76ca09:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982e000 con 0x5627b967a400 2024-09-13T23:43:52.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 18 ==== osd_op_reply(73 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:74 4.2f 4:f6499270:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982e400 con 0x5627b95b3000 2024-09-13T23:43:52.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(74 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10 [call out=697b] v0'0 uv13 ondisk = 0) v8 ==== 195+0+697 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20aa3e0f40 10 cls_bucket_list_unordered: got 24d27c169c2c881eb09a065116f2aa5c 2024-09-13T23:43:52.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20aa3e0f40 10 cls_bucket_list_unordered: got bb3aec0fdcdbc2974890f805c585d432 2024-09-13T23:43:52.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20aa3e0f40 20 get_obj_state: rctx=0x5627b92a18f0 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x5627b97b21e8 s->prefetch_data=0 2024-09-13T23:43:52.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:75 3.2s0 3:4281e5ce:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_9c8454ddf7aa50116496bac348d7550d:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982e000 con 0x5627b967a000 2024-09-13T23:43:52.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(75 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_9c8454ddf7aa50116496bac348d7550d [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20aa3e0f40 10 manifest: total_size = 6 2024-09-13T23:43:52.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20aa3e0f40 20 get_obj_state: setting s->obj_tag to 9144208c-3c85-4fb6-9007-2693d5a930f0.4222.6961131504250238250 2024-09-13T23:43:52.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20aa3e0f40 20 get_obj_state: rctx=0x5627b92a18f0 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x5627b97b21e8 s->prefetch_data=0 2024-09-13T23:43:52.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20aa3e0f40 20 prepare_atomic_modification: state is not atomic. state=0x5627b97b21e8 2024-09-13T23:43:52.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20aa3e0f40 20 bucket index object: default.rgw.buckets.index:.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2 2024-09-13T23:43:52.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.743+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 4.b 4:d3e0788f:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680800 con 0x5627b967a400 2024-09-13T23:43:52.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.747+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 19 ==== osd_op_reply(76 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2 [stat,call,call] v35'6 uv6 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.747+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 3.2s0 3:4281e5ce:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_9c8454ddf7aa50116496bac348d7550d:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680000 con 0x5627b967a000 2024-09-13T23:43:52.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.747+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(77 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_9c8454ddf7aa50116496bac348d7550d [call] v35'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 4.b 4:d3e0788f:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a400 2024-09-13T23:43:52.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20aa3e0f40 20 get_obj_state: rctx=0x5627b92a18f0 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x5627b97b21e8 s->prefetch_data=0 2024-09-13T23:43:52.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:79 3.15s0 3:abf374ee:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_b1b9a972ccd8c962a473909b97007eb4:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967a800 con 0x5627b967a400 2024-09-13T23:43:52.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 20 ==== osd_op_reply(79 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_b1b9a972ccd8c962a473909b97007eb4 [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20aa3e0f40 10 manifest: total_size = 3 2024-09-13T23:43:52.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20aa3e0f40 20 get_obj_state: setting s->obj_tag to 9144208c-3c85-4fb6-9007-2693d5a930f0.4222.9698046517513202036 2024-09-13T23:43:52.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20aa3e0f40 20 get_obj_state: rctx=0x5627b92a18f0 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x5627b97b21e8 s->prefetch_data=0 2024-09-13T23:43:52.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20aa3e0f40 20 prepare_atomic_modification: state is not atomic. state=0x5627b97b21e8 2024-09-13T23:43:52.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20aa3e0f40 20 bucket index object: default.rgw.buckets.index:.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6 2024-09-13T23:43:52.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:80 4.17 4:e8760d03:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b400 con 0x5627b967a400 2024-09-13T23:43:52.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 21 ==== osd_op_reply(78 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2 [stat,call,call] v35'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20a338e700 20 handle_completion(): completion ok for obj=9c8454ddf7aa50116496bac348d7550d 2024-09-13T23:43:52.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 22 ==== osd_op_reply(80 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6 [stat,call,call] v35'4 uv4 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.751+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:81 3.15s0 3:abf374ee:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_b1b9a972ccd8c962a473909b97007eb4:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9714800 con 0x5627b967a400 2024-09-13T23:43:52.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.755+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 23 ==== osd_op_reply(81 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_b1b9a972ccd8c962a473909b97007eb4 [call] v35'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.755+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:82 4.17 4:e8760d03:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967a800 con 0x5627b967a400 2024-09-13T23:43:52.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.755+0000 7f20aa3e0f40 20 get_obj_state: rctx=0x5627b92a18f0 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x5627b97b21e8 s->prefetch_data=0 2024-09-13T23:43:52.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.755+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:83 3.27s0 3:e5547565:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_c785e1ed2950e3e36b1e2ca01f299a54:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a000 2024-09-13T23:43:52.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.755+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(83 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_c785e1ed2950e3e36b1e2ca01f299a54 [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.755+0000 7f20aa3e0f40 10 manifest: total_size = 4 2024-09-13T23:43:52.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.755+0000 7f20aa3e0f40 20 get_obj_state: setting s->obj_tag to 9144208c-3c85-4fb6-9007-2693d5a930f0.4222.1405902904018867093 2024-09-13T23:43:52.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.755+0000 7f20aa3e0f40 20 get_obj_state: rctx=0x5627b92a18f0 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x5627b97b21e8 s->prefetch_data=0 2024-09-13T23:43:52.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.755+0000 7f20aa3e0f40 20 prepare_atomic_modification: state is not atomic. state=0x5627b97b21e8 2024-09-13T23:43:52.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.755+0000 7f20aa3e0f40 20 bucket index object: default.rgw.buckets.index:.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8 2024-09-13T23:43:52.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.755+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:84 4.0 4:0076bdb1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680800 con 0x5627b967a400 2024-09-13T23:43:52.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.759+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 24 ==== osd_op_reply(82 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6 [stat,call,call] v35'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.759+0000 7f20a2b8d700 20 handle_completion(): completion ok for obj=b1b9a972ccd8c962a473909b97007eb4 2024-09-13T23:43:52.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.759+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 25 ==== osd_op_reply(84 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8 [stat,call,call] v35'4 uv4 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.759+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:85 3.27s0 3:e5547565:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_c785e1ed2950e3e36b1e2ca01f299a54:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982fc00 con 0x5627b967a000 2024-09-13T23:43:52.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(85 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_c785e1ed2950e3e36b1e2ca01f299a54 [call] v35'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:86 4.0 4:0076bdb1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982f400 con 0x5627b967a400 2024-09-13T23:43:52.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20aa3e0f40 20 get_obj_state: rctx=0x5627b92a18f0 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x5627b97b21e8 s->prefetch_data=0 2024-09-13T23:43:52.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:87 3.10s0 3:0acc8ce2:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_24d27c169c2c881eb09a065116f2aa5c:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982f000 con 0x5627b95b3400 2024-09-13T23:43:52.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(87 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_24d27c169c2c881eb09a065116f2aa5c [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 262+0+893 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20aa3e0f40 10 manifest: total_size = 5 2024-09-13T23:43:52.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20aa3e0f40 20 get_obj_state: setting s->obj_tag to 9144208c-3c85-4fb6-9007-2693d5a930f0.4222.1234671231680772488 2024-09-13T23:43:52.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20aa3e0f40 20 get_obj_state: rctx=0x5627b92a18f0 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x5627b97b21e8 s->prefetch_data=0 2024-09-13T23:43:52.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20aa3e0f40 20 prepare_atomic_modification: state is not atomic. state=0x5627b97b21e8 2024-09-13T23:43:52.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20aa3e0f40 20 bucket index object: default.rgw.buckets.index:.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10 2024-09-13T23:43:52.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:88 4.2f 4:f6499270:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982ec00 con 0x5627b95b3000 2024-09-13T23:43:52.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 26 ==== osd_op_reply(86 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8 [stat,call,call] v35'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20a338e700 20 handle_completion(): completion ok for obj=c785e1ed2950e3e36b1e2ca01f299a54 2024-09-13T23:43:52.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(88 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10 [stat,call,call] v35'14 uv14 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.763+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:89 3.10s0 3:0acc8ce2:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_24d27c169c2c881eb09a065116f2aa5c:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982e800 con 0x5627b95b3400 2024-09-13T23:43:52.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.767+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(89 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_24d27c169c2c881eb09a065116f2aa5c [call] v35'2 uv2 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.767+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:90 4.2f 4:f6499270:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97ba000 con 0x5627b95b3000 2024-09-13T23:43:52.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.767+0000 7f20aa3e0f40 20 get_obj_state: rctx=0x5627b92a18f0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x5627b97b21e8 s->prefetch_data=0 2024-09-13T23:43:52.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.767+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:91 3.1fs0 3:fb2b0a4f:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97ba400 con 0x5627b95b3000 2024-09-13T23:43:52.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.767+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(91 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=909b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+925 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.767+0000 7f20aa3e0f40 10 manifest: total_size = 5 2024-09-13T23:43:52.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.767+0000 7f20aa3e0f40 20 get_obj_state: setting s->obj_tag to _CtZaybV8LtsgmEBRAh2kNBN-wLlFOcF 2024-09-13T23:43:52.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.767+0000 7f20aa3e0f40 20 get_obj_state: rctx=0x5627b92a18f0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x5627b97b21e8 s->prefetch_data=0 2024-09-13T23:43:52.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.767+0000 7f20aa3e0f40 20 prepare_atomic_modification: state is not atomic. state=0x5627b97b21e8 2024-09-13T23:43:52.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.767+0000 7f20aa3e0f40 20 bucket index object: default.rgw.buckets.index:.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10 2024-09-13T23:43:52.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.767+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:92 4.2f 4:f6499270:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97ba800 con 0x5627b95b3000 2024-09-13T23:43:52.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.771+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(90 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10 [stat,call,call] v35'15 uv15 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.771+0000 7f20a2b8d700 20 handle_completion(): completion ok for obj=24d27c169c2c881eb09a065116f2aa5c 2024-09-13T23:43:52.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.771+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(92 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10 [stat,call,call] v35'16 uv16 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.780 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.771+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:93 3.1fs0 3:fb2b0a4f:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_bb3aec0fdcdbc2974890f805c585d432:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bac00 con 0x5627b95b3000 2024-09-13T23:43:52.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 28 ==== osd_op_reply(93 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5_bb3aec0fdcdbc2974890f805c585d432 [call] v35'4 uv4 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:94 4.2f 4:f6499270:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bb400 con 0x5627b95b3000 2024-09-13T23:43:52.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 20 list_objects_ordered: starting attempt 1 2024-09-13T23:43:52.783 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-09-13T23:43:52.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:95 4.2 4:42118f18:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bb800 con 0x5627b95b3000 2024-09-13T23:43:52.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:96 4.36 4:6f89048c:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bbc00 con 0x5627b95b3000 2024-09-13T23:43:52.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:97 4.b 4:d3e0788f:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982f400 con 0x5627b967a400 2024-09-13T23:43:52.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:98 4.14 4:2b4eeb18:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982ec00 con 0x5627b967a000 2024-09-13T23:43:52.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:99 4.9 4:92fa94e2:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982e400 con 0x5627b95b3000 2024-09-13T23:43:52.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:100 4.c 4:3265cd07:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9714800 con 0x5627b95b3000 2024-09-13T23:43:52.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:101 4.17 4:e8760d03:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715800 con 0x5627b967a400 2024-09-13T23:43:52.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:102 4.3c 4:3e7ace4d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97ba000 con 0x5627b95b3000 2024-09-13T23:43:52.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:103 4.0 4:0076bdb1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97ba400 con 0x5627b967a400 2024-09-13T23:43:52.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:104 4.33 4:cc76ca09:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97ba800 con 0x5627b967a400 2024-09-13T23:43:52.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(98 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.3 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:105 4.2f 4:f6499270:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bac00 con 0x5627b95b3000 2024-09-13T23:43:52.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 27 ==== osd_op_reply(97 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 28 ==== osd_op_reply(101 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 29 ==== osd_op_reply(103 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 30 ==== osd_op_reply(104 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 29 ==== osd_op_reply(95 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 30 ==== osd_op_reply(99 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 31 ==== osd_op_reply(96 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 32 ==== osd_op_reply(100 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.5 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 33 ==== osd_op_reply(102 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 34 ==== osd_op_reply(94 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10 [stat,call,call] v35'17 uv17 ondisk = 0) v8 ==== 279+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a338e700 20 handle_completion(): completion ok for obj=bb3aec0fdcdbc2974890f805c585d432 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.775+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 35 ==== osd_op_reply(105 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10 [call out=124b] v0'0 uv17 ondisk = 0) v8 ==== 195+0+124 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5])"; uploads.size()=0; is_truncated=0 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5]) 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x17, cached=0x11) 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 20 lifecycle: rados->read ofs=0 len=0 2024-09-13T23:43:52.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:106 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bb400 con 0x5627b967a400 2024-09-13T23:43:52.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 31 ==== osd_op_reply(106 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv33 ondisk = 0) v8 ==== 275+0+304 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 20 lifecycle: rados_obj.operate() r=0 bl.length=165 2024-09-13T23:43:52.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-09-13T23:43:52.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 10 lifecycle: moving default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:43:52.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:52.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:107 8.2 8:444f11e7:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b400 con 0x5627b967a400 2024-09-13T23:43:52.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 32 ==== osd_op_reply(107 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 [delete,create,call,call,writefull 0~352,setxattr (123)] v35'4 uv4 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 info.flags=0x17 2024-09-13T23:43:52.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 to cache LRU end 2024-09-13T23:43:52.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:52.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 10 lifecycle: distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5, ofs0, ns] 2024-09-13T23:43:52.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:108 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94728614342016 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982f400 con 0x5627b95b3400 2024-09-13T23:43:52.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== watch-notify(notify (1) cookie 94728612666112 notify 150323855396 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x5627b956a680 con 0x5627b95b3400 2024-09-13T23:43:52.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20a338e700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855396 cookie 94728612666112 notifier 4702 bl.length()=725 2024-09-13T23:43:52.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(108 notify.0 [notify cookie 94728614342016 out=8b] v0'0 uv177 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20a338e700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 info.flags=0x17 2024-09-13T23:43:52.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20a338e700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 to cache LRU end 2024-09-13T23:43:52.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20a338e700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:43:52.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.779+0000 7f20a338e700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:109 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3c00 con 0x5627b95b3400 2024-09-13T23:43:52.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.783+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(109 notify.0 [notify-ack] v0'0 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.783+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== watch-notify(notify_complete (2) cookie 94728614342016 notify 150323855396 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5627b956a820 con 0x5627b95b3400 2024-09-13T23:43:52.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.783+0000 7f20aa3e0f40 20 lifecycle: handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5]): orig_sources= new_sources= 2024-09-13T23:43:52.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.783+0000 7f20aa3e0f40 20 lifecycle: handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5]): potential sources added=[] removed=[] 2024-09-13T23:43:52.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.783+0000 7f20aa3e0f40 20 lifecycle: handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5]): orig_dests= new_dests= 2024-09-13T23:43:52.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.783+0000 7f20aa3e0f40 20 lifecycle: handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5]): potential dests added=[] removed=[] 2024-09-13T23:43:52.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.783+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 6.15 6:abd89f08:lc::lc.23:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982e400 con 0x5627b9681400 2024-09-13T23:43:52.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.783+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(33 lc.23 [call] v35'25 uv25 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.783+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 6.15 6:abd89f08:lc::lc.23:head [call rgw.lc_rm_entry in=92b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982f800 con 0x5627b9681400 2024-09-13T23:43:52.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.787+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(34 lc.23 [call] v35'26 uv26 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.787+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:35 6.15 6:abd89f08:lc::lc.23:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982e000 con 0x5627b9681400 2024-09-13T23:43:52.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(35 lc.23 [call] v35'27 uv27 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b9681400 2024-09-13T23:43:52.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:110 4.2 4:42118f18:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982fc00 con 0x5627b95b3000 2024-09-13T23:43:52.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:111 4.36 4:6f89048c:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982ec00 con 0x5627b95b3000 2024-09-13T23:43:52.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:112 4.b 4:d3e0788f:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b95b3800 con 0x5627b967a400 2024-09-13T23:43:52.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:113 4.14 4:2b4eeb18:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9681800 con 0x5627b967a000 2024-09-13T23:43:52.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:114 4.9 4:92fa94e2:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680000 con 0x5627b95b3000 2024-09-13T23:43:52.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:115 4.c 4:3265cd07:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967a800 con 0x5627b95b3000 2024-09-13T23:43:52.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:116 4.17 4:e8760d03:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9714800 con 0x5627b967a400 2024-09-13T23:43:52.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:117 4.3c 4:3e7ace4d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715800 con 0x5627b95b3000 2024-09-13T23:43:52.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:118 4.0 4:0076bdb1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982f000 con 0x5627b967a400 2024-09-13T23:43:52.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:119 4.33 4:cc76ca09:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982e800 con 0x5627b967a400 2024-09-13T23:43:52.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:120 4.2f 4:f6499270:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982f400 con 0x5627b95b3000 2024-09-13T23:43:52.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(113 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.3 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 36 ==== osd_op_reply(110 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.0 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 37 ==== osd_op_reply(111 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.1 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 38 ==== osd_op_reply(114 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 39 ==== osd_op_reply(115 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.5 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 40 ==== osd_op_reply(120 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10 [call out=124b] v0'0 uv17 ondisk = 0) v8 ==== 195+0+124 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 33 ==== osd_op_reply(112 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 34 ==== osd_op_reply(116 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 35 ==== osd_op_reply(118 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 41 ==== osd_op_reply(117 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 36 ==== osd_op_reply(119 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.9 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.791+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:121 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715c00 con 0x5627b967a400 2024-09-13T23:43:52.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 37 ==== osd_op_reply(121 foo.buckets [call] v35'36 uv36 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20aa3e0f40 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x17) 2024-09-13T23:43:52.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:122 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715400 con 0x5627b967a400 2024-09-13T23:43:52.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 38 ==== osd_op_reply(122 myfoo [call,call,delete] v35'37 uv37 ondisk = 0) v8 ==== 233+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20aa3e0f40 10 removing default.rgw.meta+root+myfoo from cache 2024-09-13T23:43:52.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20aa3e0f40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-13T23:43:52.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:123 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94728614342016 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715000 con 0x5627b967a000 2024-09-13T23:43:52.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== watch-notify(notify (1) cookie 94728612673024 notify 150323855372 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x5627b96e0b60 con 0x5627b967a000 2024-09-13T23:43:52.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20a2b8d700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855372 cookie 94728612673024 notifier 4702 bl.length()=149 2024-09-13T23:43:52.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(123 notify.6 [notify cookie 94728614342016 out=8b] v0'0 uv177 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20a2b8d700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:124 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680800 con 0x5627b967a000 2024-09-13T23:43:52.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(124 notify.6 [notify-ack] v0'0 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.795+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== watch-notify(notify_complete (2) cookie 94728614342016 notify 150323855372 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5627b96e0d00 con 0x5627b967a000 2024-09-13T23:43:52.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.799+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:125 8.2 8:444f11e7:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9714800 con 0x5627b967a400 2024-09-13T23:43:52.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.799+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 39 ==== osd_op_reply(125 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 [call,call,delete] v35'5 uv5 ondisk = 0) v8 ==== 290+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.799+0000 7f20aa3e0f40 10 removing default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 from cache 2024-09-13T23:43:52.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.799+0000 7f20aa3e0f40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5, ofs0, ns] 2024-09-13T23:43:52.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.799+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:126 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94728614342016 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715800 con 0x5627b95b3400 2024-09-13T23:43:52.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.799+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== watch-notify(notify (1) cookie 94728612666112 notify 150323855397 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x5627b956a820 con 0x5627b95b3400 2024-09-13T23:43:52.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.799+0000 7f20a338e700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855397 cookie 94728612666112 notifier 4702 bl.length()=206 2024-09-13T23:43:52.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.799+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(126 notify.0 [notify cookie 94728614342016 out=8b] v0'0 uv177 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.799+0000 7f20a338e700 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:127 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967ac00 con 0x5627b95b3400 2024-09-13T23:43:52.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== watch-notify(notify_complete (2) cookie 94728614342016 notify 150323855397 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5627b956a9c0 con 0x5627b95b3400 2024-09-13T23:43:52.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(127 notify.0 [notify-ack] v0'0 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:128 4.2 4:42118f18:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9715c00 con 0x5627b95b3000 2024-09-13T23:43:52.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:129 4.36 4:6f89048c:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982fc00 con 0x5627b95b3000 2024-09-13T23:43:52.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:130 4.b 4:d3e0788f:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982ec00 con 0x5627b967a400 2024-09-13T23:43:52.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:131 4.14 4:2b4eeb18:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982f000 con 0x5627b967a000 2024-09-13T23:43:52.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:132 4.9 4:92fa94e2:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982f400 con 0x5627b95b3000 2024-09-13T23:43:52.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:133 4.c 4:3265cd07:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b982e800 con 0x5627b95b3000 2024-09-13T23:43:52.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:134 4.17 4:e8760d03:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b400 con 0x5627b967a400 2024-09-13T23:43:52.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:135 4.3c 4:3e7ace4d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967a800 con 0x5627b95b3000 2024-09-13T23:43:52.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:136 4.0 4:0076bdb1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680000 con 0x5627b967a400 2024-09-13T23:43:52.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:137 4.33 4:cc76ca09:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97ba000 con 0x5627b967a400 2024-09-13T23:43:52.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:138 4.2f 4:f6499270:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97ba400 con 0x5627b95b3000 2024-09-13T23:43:52.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.803+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 42 ==== osd_op_reply(128 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.0 [delete] v35'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 43 ==== osd_op_reply(129 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.1 [delete] v35'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 44 ==== osd_op_reply(133 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.5 [delete] v35'10 uv10 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 45 ==== osd_op_reply(132 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.4 [delete] v35'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 25 ==== osd_op_reply(131 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.3 [delete] v35'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 46 ==== osd_op_reply(135 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.7 [delete] v35'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 40 ==== osd_op_reply(137 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.9 [delete] v35'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 41 ==== osd_op_reply(130 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.2 [delete] v35'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 42 ==== osd_op_reply(134 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.6 [delete] v35'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 47 ==== osd_op_reply(138 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.10 [delete] v35'18 uv18 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 43 ==== osd_op_reply(136 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5.8 [delete] v35'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20aa3e0f40 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 : miss 2024-09-13T23:43:52.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20aa3e0f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:139 6.d 6:b5fb3a1c:::pubsub..bucket.myfoo%2f9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97ba800 con 0x5627b95b3400 2024-09-13T23:43:52.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(139 pubsub..bucket.myfoo/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20aa3e0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20aa3e0f40 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 info.flags=0x0 2024-09-13T23:43:52.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20aa3e0f40 10 adding default.rgw.log++pubsub..bucket.myfoo/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 to cache LRU end 2024-09-13T23:43:52.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.807+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:140 6.d 6:b5fb3a1c:::pubsub..bucket.myfoo%2f9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bac00 con 0x5627b95b3400 2024-09-13T23:43:52.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.811+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 31 ==== osd_op_reply(140 pubsub..bucket.myfoo/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.5 [delete] v35'33 uv29 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.811+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:141 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bb000 con 0x5627b967a400 2024-09-13T23:43:52.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.811+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 44 ==== osd_op_reply(141 foo.buckets [call] v35'38 uv38 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.815+0000 7f208fb67700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:52.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.815+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:142 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94728614338560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bb800 con 0x5627b95b3000 2024-09-13T23:43:52.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.815+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 48 ==== osd_op_reply(142 data_loggenerations_metadata [watch unwatch cookie 94728614338560] v35'186 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.815+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:143 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94728612666112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bbc00 con 0x5627b95b3400 2024-09-13T23:43:52.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.819+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 32 ==== osd_op_reply(143 notify.0 [watch unwatch cookie 94728612666112] v35'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.819+0000 7f20aa3e0f40 20 remove_watcher() i=0 2024-09-13T23:43:52.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.819+0000 7f20aa3e0f40 2 removed watcher, disabling cache 2024-09-13T23:43:52.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.819+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:144 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94728612667264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97ba000 con 0x5627b95b3000 2024-09-13T23:43:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.819+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.1 v2:172.21.15.167:6800/3875661733 49 ==== osd_op_reply(144 notify.1 [watch unwatch cookie 94728612667264] v35'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b95b3000 2024-09-13T23:43:52.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.819+0000 7f20aa3e0f40 20 remove_watcher() i=1 2024-09-13T23:43:52.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.819+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:145 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94728612668416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97ba400 con 0x5627b95b3400 2024-09-13T23:43:52.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.823+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 33 ==== osd_op_reply(145 notify.2 [watch unwatch cookie 94728612668416] v35'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.823+0000 7f20aa3e0f40 20 remove_watcher() i=2 2024-09-13T23:43:52.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.823+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:146 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94728612669568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bb000 con 0x5627b967a400 2024-09-13T23:43:52.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.823+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 45 ==== osd_op_reply(146 notify.3 [watch unwatch cookie 94728612669568] v35'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.823+0000 7f20aa3e0f40 20 remove_watcher() i=3 2024-09-13T23:43:52.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.823+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:147 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94728612670720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bb800 con 0x5627b95b3400 2024-09-13T23:43:52.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.827+0000 7f20a8b99700 1 -- 172.21.15.167:0/2161935347 <== osd.0 v2:172.21.15.12:6800/2649399468 34 ==== osd_op_reply(147 notify.4 [watch unwatch cookie 94728612670720] v35'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659200 con 0x5627b95b3400 2024-09-13T23:43:52.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.827+0000 7f20aa3e0f40 20 remove_watcher() i=4 2024-09-13T23:43:52.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.827+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:148 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94728612671872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b97bb400 con 0x5627b967a000 2024-09-13T23:43:52.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.831+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 26 ==== osd_op_reply(148 notify.5 [watch unwatch cookie 94728612671872] v35'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.831+0000 7f20aa3e0f40 20 remove_watcher() i=5 2024-09-13T23:43:52.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.831+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:149 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94728612673024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b9680000 con 0x5627b967a000 2024-09-13T23:43:52.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.831+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 <== osd.3 v2:172.21.15.167:6816/3688363524 27 ==== osd_op_reply(149 notify.6 [watch unwatch cookie 94728612673024] v35'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9659440 con 0x5627b967a000 2024-09-13T23:43:52.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.831+0000 7f20aa3e0f40 20 remove_watcher() i=6 2024-09-13T23:43:52.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.831+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:150 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94728612674176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5627b967b400 con 0x5627b967a400 2024-09-13T23:43:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20a939a700 1 -- 172.21.15.167:0/2161935347 <== osd.2 v2:172.21.15.167:6808/1180369139 46 ==== osd_op_reply(150 notify.7 [watch unwatch cookie 94728612674176] v35'179 uv177 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5627b9658b40 con 0x5627b967a400 2024-09-13T23:43:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 20 remove_watcher() i=7 2024-09-13T23:43:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5627b95b3400 msgr2=0x5627b94f8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5627b95b3400 0x5627b94f8100 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5627b95b3000 msgr2=0x5627b94f7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5627b95b3000 0x5627b94f7b80 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5627b967a400 msgr2=0x5627b94f8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5627b967a400 0x5627b94f8c00 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5627b967a000 msgr2=0x5627b94f8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5627b967a000 0x5627b94f8680 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b95b2c00 msgr2=0x5627b94f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b95b2c00 0x5627b94f7600 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x5627b95b5b30 tx=0x5627b966a0c0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 msgr2=0x5627b94f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2161935347 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211800 0x5627b94f6b00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x5627ba084300 tx=0x5627b95765a0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 reap_dead start 2024-09-13T23:43:52.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20a8398700 1 -- 172.21.15.167:0/2161935347 reap_dead start 2024-09-13T23:43:52.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 shutdown_connections 2024-09-13T23:43:52.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 >> 172.21.15.167:0/2161935347 conn(0x5627b9211400 msgr2=0x5627b9311800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 shutdown_connections 2024-09-13T23:43:52.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2161935347 wait complete. 2024-09-13T23:43:52.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5627b9681400 msgr2=0x5627b971a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5627b9681400 0x5627b971a580 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5627b9714400 msgr2=0x5627b971b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5627b9714400 0x5627b971b600 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5627b9680400 msgr2=0x5627b971ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5627b9680400 0x5627b971ab00 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5627b967b000 msgr2=0x5627b971b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5627b967b000 0x5627b971b080 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b9714000 msgr2=0x5627b971a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b9714000 0x5627b971a000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x5627ba084f00 tx=0x5627b9577500 comp rx=0 tx=0).stop 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9681000 msgr2=0x5627b94f9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9681000 0x5627b94f9700 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x5627b96902a0 tx=0x5627b966a180 comp rx=0 tx=0).stop 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20a8b99700 1 -- 172.21.15.167:0/2222944702 reap_dead start 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 shutdown_connections 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/2222944702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9681000 0x5627b94f9700 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 >> 172.21.15.167:0/2222944702 conn(0x5627b9681c00 msgr2=0x5627b9312000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 shutdown_connections 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/2222944702 wait complete. 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3834912768 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b95b2000 msgr2=0x5627b94f7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/3834912768 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b95b2000 0x5627b94f7080 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x5627ba0850b0 tx=0x5627b9576b40 comp rx=0 tx=0).stop 2024-09-13T23:43:52.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3834912768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 msgr2=0x5627b94f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/3834912768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x5627ba0716b0 tx=0x5627b9315920 comp rx=0 tx=0).stop 2024-09-13T23:43:52.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3834912768 shutdown_connections 2024-09-13T23:43:52.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/3834912768 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5627b95b2000 0x5627b94f7080 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 --2- 172.21.15.167:0/3834912768 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5627b9211000 0x5627b94f6580 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3834912768 >> 172.21.15.167:0/3834912768 conn(0x5627b9210c00 msgr2=0x5627b9311000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3834912768 shutdown_connections 2024-09-13T23:43:52.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.835+0000 7f20aa3e0f40 1 -- 172.21.15.167:0/3834912768 wait complete. 2024-09-13T23:43:52.851 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-09-13T23:43:52.851 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-09-13T23:43:52.851 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid foo --caps user=read 2024-09-13T23:43:52.895 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:52.895 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:52.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.887+0000 7f0c87419f40 1 Processor -- start 2024-09-13T23:43:52.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.887+0000 7f0c87419f40 1 -- start start 2024-09-13T23:43:52.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.887+0000 7f0c87419f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55e1a67c1000 0x55e1a6aa6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.887+0000 7f0c87419f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e1a6885500 con 0x55e1a67c1400 2024-09-13T23:43:52.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.887+0000 7f0c87419f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e1a684ea00 con 0x55e1a67c1000 2024-09-13T23:43:52.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.887+0000 7f0c85bd2700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55e1a67c1000 0x55e1a6aa6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.887+0000 7f0c853d1700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55e1a67c1400 0x55e1a68c0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:49370/0 (socket says 172.21.15.167:49370) 2024-09-13T23:43:52.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.887+0000 7f0c853d1700 1 -- 172.21.15.167:0/3998036659 learned_addr learned my addr 172.21.15.167:0/3998036659 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:52.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.887+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3998036659 >> v1:172.21.15.12:6789/0 conn(0x55e1a67c1400 legacy=0x55e1a68c0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.887+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3998036659 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1a67d3d40 con 0x55e1a67c1000 2024-09-13T23:43:52.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.887+0000 7f0c85bd2700 1 --2- 172.21.15.167:0/3998036659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55e1a6aa1a70 tx=0x55e1a68c5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e5e5786ce24eae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c84bd0700 1 -- 172.21.15.167:0/3998036659 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a68654a0 con 0x55e1a67c1000 2024-09-13T23:43:52.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c84bd0700 1 -- 172.21.15.167:0/3998036659 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1a6885800 con 0x55e1a67c1000 2024-09-13T23:43:52.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c84bd0700 1 -- 172.21.15.167:0/3998036659 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a6865600 con 0x55e1a67c1000 2024-09-13T23:43:52.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c87419f40 1 -- 172.21.15.167:0/3998036659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 msgr2=0x55e1a6aa6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3998036659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55e1a6aa1a70 tx=0x55e1a68c5920 comp rx=0 tx=0).stop 2024-09-13T23:43:52.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c87419f40 1 -- 172.21.15.167:0/3998036659 shutdown_connections 2024-09-13T23:43:52.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3998036659 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c87419f40 1 -- 172.21.15.167:0/3998036659 >> 172.21.15.167:0/3998036659 conn(0x55e1a67c0c00 msgr2=0x55e1a68c0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c87419f40 1 -- 172.21.15.167:0/3998036659 shutdown_connections 2024-09-13T23:43:52.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c87419f40 1 -- 172.21.15.167:0/3998036659 wait complete. 2024-09-13T23:43:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c87419f40 1 Processor -- start 2024-09-13T23:43:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c87419f40 1 -- start start 2024-09-13T23:43:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c87419f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c87419f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a684e500 con 0x55e1a67c1000 2024-09-13T23:43:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c85bd2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c85bd2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52500/0 (socket says 172.21.15.167:52500) 2024-09-13T23:43:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3874028989 learned_addr learned my addr 172.21.15.167:0/3874028989 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3874028989 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1a67d3ba0 con 0x55e1a67c1000 2024-09-13T23:43:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c85bd2700 1 --2- 172.21.15.167:0/3874028989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55e1a762f800 tx=0x55e1a68c5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498e9b20bc9bcaf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c843cf700 1 -- 172.21.15.167:0/3874028989 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a6865760 con 0x55e1a67c1000 2024-09-13T23:43:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c843cf700 1 -- 172.21.15.167:0/3874028989 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1a6885b00 con 0x55e1a67c1000 2024-09-13T23:43:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.891+0000 7f0c843cf700 1 -- 172.21.15.167:0/3874028989 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a68651e0 con 0x55e1a67c1000 2024-09-13T23:43:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 -- 172.21.15.167:0/3874028989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 msgr2=0x55e1a6aa6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3874028989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x55e1a762f800 tx=0x55e1a68c5800 comp rx=0 tx=0).stop 2024-09-13T23:43:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 -- 172.21.15.167:0/3874028989 shutdown_connections 2024-09-13T23:43:52.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3874028989 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 -- 172.21.15.167:0/3874028989 >> 172.21.15.167:0/3874028989 conn(0x55e1a67c0c00 msgr2=0x55e1a68c1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 -- 172.21.15.167:0/3874028989 shutdown_connections 2024-09-13T23:43:52.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 -- 172.21.15.167:0/3874028989 wait complete. 2024-09-13T23:43:52.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 Processor -- start 2024-09-13T23:43:52.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 -- start start 2024-09-13T23:43:52.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a684e3c0 con 0x55e1a67c1000 2024-09-13T23:43:52.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c85bd2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c85bd2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52516/0 (socket says 172.21.15.167:52516) 2024-09-13T23:43:52.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c85bd2700 1 -- 172.21.15.167:0/2441703037 learned_addr learned my addr 172.21.15.167:0/2441703037 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c85bd2700 1 -- 172.21.15.167:0/2441703037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1a67d3a00 con 0x55e1a67c1000 2024-09-13T23:43:52.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c85bd2700 1 --2- 172.21.15.167:0/2441703037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x55e1a762f6b0 tx=0x55e1a68c5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c82bcc700 1 -- 172.21.15.167:0/2441703037 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a68654a0 con 0x55e1a67c1000 2024-09-13T23:43:52.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c82bcc700 1 -- 172.21.15.167:0/2441703037 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1a6885c80 con 0x55e1a67c1000 2024-09-13T23:43:52.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 -- 172.21.15.167:0/2441703037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1a67d3860 con 0x55e1a67c1000 2024-09-13T23:43:52.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c82bcc700 1 -- 172.21.15.167:0/2441703037 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a6865600 con 0x55e1a67c1000 2024-09-13T23:43:52.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.895+0000 7f0c87419f40 1 -- 172.21.15.167:0/2441703037 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1a67d36c0 con 0x55e1a67c1000 2024-09-13T23:43:52.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c87419f40 1 Processor -- start 2024-09-13T23:43:52.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c87419f40 1 -- start start 2024-09-13T23:43:52.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c87419f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 0x55e1a6aa6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c87419f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a67e6640 con 0x55e1a67c1800 2024-09-13T23:43:52.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c82bcc700 1 -- 172.21.15.167:0/2441703037 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e1a68bab00 con 0x55e1a67c1000 2024-09-13T23:43:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c82bcc700 1 --2- 172.21.15.167:0/2441703037 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6b6a000 0x55e1a6aa7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c82bcc700 1 -- 172.21.15.167:0/2441703037 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55e1a6827180 con 0x55e1a67c1000 2024-09-13T23:43:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c863d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 0x55e1a6aa6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c85bd2700 1 --2- 172.21.15.167:0/2441703037 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6b6a000 0x55e1a6aa7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c863d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 0x55e1a6aa6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52530/0 (socket says 172.21.15.167:52530) 2024-09-13T23:43:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c863d3700 1 -- 172.21.15.167:0/3672541007 learned_addr learned my addr 172.21.15.167:0/3672541007 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c863d3700 1 -- 172.21.15.167:0/3672541007 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1a67d2000 con 0x55e1a67c1800 2024-09-13T23:43:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c863d3700 1 --2- 172.21.15.167:0/3672541007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 0x55e1a6aa6b00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x55e1a6af2420 tx=0x55e1a6b2ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bffe014a7daa03 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c803c7700 1 -- 172.21.15.167:0/3672541007 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a68654a0 con 0x55e1a67c1800 2024-09-13T23:43:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c803c7700 1 -- 172.21.15.167:0/3672541007 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1a6b7a000 con 0x55e1a67c1800 2024-09-13T23:43:52.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c803c7700 1 -- 172.21.15.167:0/3672541007 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a6865600 con 0x55e1a67c1800 2024-09-13T23:43:52.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c85bd2700 1 --2- 172.21.15.167:0/2441703037 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6b6a000 0x55e1a6aa7080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55e1a6af2f90 tx=0x55e1a6b2eb40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.899+0000 7f0c87419f40 1 -- 172.21.15.167:0/3672541007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 msgr2=0x55e1a6aa6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3672541007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 0x55e1a6aa6b00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x55e1a6af2420 tx=0x55e1a6b2ee40 comp rx=0 tx=0).stop 2024-09-13T23:43:52.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c87419f40 1 -- 172.21.15.167:0/3672541007 shutdown_connections 2024-09-13T23:43:52.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3672541007 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 0x55e1a6aa6b00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c87419f40 1 -- 172.21.15.167:0/3672541007 >> 172.21.15.167:0/3672541007 conn(0x55e1a67c1400 msgr2=0x55e1a68c1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c87419f40 1 -- 172.21.15.167:0/3672541007 shutdown_connections 2024-09-13T23:43:52.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c87419f40 1 -- 172.21.15.167:0/3672541007 wait complete. 2024-09-13T23:43:52.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c87419f40 1 Processor -- start 2024-09-13T23:43:52.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c87419f40 1 -- start start 2024-09-13T23:43:52.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c87419f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 0x55e1a6aa6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c87419f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a67e68c0 con 0x55e1a67c1800 2024-09-13T23:43:52.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c863d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 0x55e1a6aa6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c863d3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 0x55e1a6aa6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52538/0 (socket says 172.21.15.167:52538) 2024-09-13T23:43:52.913 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 learned_addr learned my addr 172.21.15.167:0/3391884987 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1a67d2ea0 con 0x55e1a67c1800 2024-09-13T23:43:52.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c863d3700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 0x55e1a6aa6b00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55e1a6af21e0 tx=0x55e1a6b2ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c7ebc4700 1 -- 172.21.15.167:0/3391884987 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a6865760 con 0x55e1a67c1800 2024-09-13T23:43:52.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c7ebc4700 1 -- 172.21.15.167:0/3391884987 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1a6b7a180 con 0x55e1a67c1800 2024-09-13T23:43:52.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1a67d2820 con 0x55e1a67c1800 2024-09-13T23:43:52.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c7ebc4700 1 -- 172.21.15.167:0/3391884987 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a68654a0 con 0x55e1a67c1800 2024-09-13T23:43:52.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.903+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1a67d29c0 con 0x55e1a67c1800 2024-09-13T23:43:52.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c7ebc4700 1 -- 172.21.15.167:0/3391884987 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e1a68badc0 con 0x55e1a67c1800 2024-09-13T23:43:52.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c7ebc4700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6b6ac00 0x55e1a6aa7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c7ebc4700 1 -- 172.21.15.167:0/3391884987 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55e1a6827340 con 0x55e1a67c1800 2024-09-13T23:43:52.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c853d1700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6b6ac00 0x55e1a6aa7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c87419f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e1a6b6b000 0x55e1a6aa7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b400 con 0x55e1a6b6b000 2024-09-13T23:43:52.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c863d3700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e1a6b6b000 0x55e1a6aa7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c853d1700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6b6ac00 0x55e1a6aa7600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55e1a6b6de60 tx=0x55e1a6c140c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c863d3700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e1a6b6b000 0x55e1a6aa7b80 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c87419f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c87419f40 20 realm 2024-09-13T23:43:52.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c87419f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b000 2024-09-13T23:43:52.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c87419f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c87419f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:52.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c87419f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6bc00 con 0x55e1a6b6b000 2024-09-13T23:43:52.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.907+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c87419f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c87419f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.919 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e1a6b6b400 0x55e1a6aa8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b400 2024-09-13T23:43:52.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c85bd2700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e1a6b6b400 0x55e1a6aa8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c85bd2700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e1a6b6b400 0x55e1a6aa8100 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c87419f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:52.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c87419f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6bc00 con 0x55e1a6b6b400 2024-09-13T23:43:52.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c87419f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:52.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c87419f40 20 searching for the correct realm 2024-09-13T23:43:52.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e1a6c32000 0x55e1a6aa8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c32400 con 0x55e1a6c32000 2024-09-13T23:43:52.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c853d1700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e1a6c32000 0x55e1a6aa8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.911+0000 7f0c853d1700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e1a6c32000 0x55e1a6aa8680 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c32400 con 0x55e1a6b6b400 2024-09-13T23:43:52.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c85bd2700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e1a6c32400 0x55e1a6aa8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32400 2024-09-13T23:43:52.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c863d3700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e1a6c32400 0x55e1a6aa8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.924 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c863d3700 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e1a6c32400 0x55e1a6aa8c00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:52.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b000 2024-09-13T23:43:52.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.925 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b000 2024-09-13T23:43:52.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.915+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32000 2024-09-13T23:43:52.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.919+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.919+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b000 2024-09-13T23:43:52.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.919+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.927 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.919+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b400 2024-09-13T23:43:52.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.919+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.919+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b400 2024-09-13T23:43:52.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.919+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.919+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b000 2024-09-13T23:43:52.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.919+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.919+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32400 2024-09-13T23:43:52.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.919+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:52.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.919+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b000 2024-09-13T23:43:52.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.923+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.923+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32400 2024-09-13T23:43:52.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.923+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:52.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.923+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b400 2024-09-13T23:43:52.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.923+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.923+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32000 2024-09-13T23:43:52.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.923+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.923+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b000 2024-09-13T23:43:52.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.923+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.923+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b400 2024-09-13T23:43:52.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32000 2024-09-13T23:43:52.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b000 2024-09-13T23:43:52.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32400 2024-09-13T23:43:52.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:52.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b000 2024-09-13T23:43:52.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32000 2024-09-13T23:43:52.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32400 2024-09-13T23:43:52.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:52.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.927+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b000 2024-09-13T23:43:52.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.931+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.938 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.931+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32400 2024-09-13T23:43:52.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.931+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:52.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.931+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32000 2024-09-13T23:43:52.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.931+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.931+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32400 2024-09-13T23:43:52.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.931+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:52.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.931+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b400 2024-09-13T23:43:52.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.931+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.931+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b400 2024-09-13T23:43:52.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.931+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.931+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32000 2024-09-13T23:43:52.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b400 2024-09-13T23:43:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b400 2024-09-13T23:43:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c87419f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:52.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c87419f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c87419f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c87419f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c87419f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c32800 con 0x55e1a6b6b000 2024-09-13T23:43:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c87419f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c87419f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c32c00 con 0x55e1a6b6b400 2024-09-13T23:43:52.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c87419f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:52.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c87419f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.935+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33000 con 0x55e1a6b6b400 2024-09-13T23:43:52.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:52.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 20 zone default found 2024-09-13T23:43:52.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 4 Realm: () 2024-09-13T23:43:52.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:52.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:52.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:52.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 20 zonegroup default 2024-09-13T23:43:52.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33400 con 0x55e1a6c32400 2024-09-13T23:43:52.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:52.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33800 con 0x55e1a6c32000 2024-09-13T23:43:52.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33c00 con 0x55e1a6c32000 2024-09-13T23:43:52.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:52.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 20 started sync module instance, tier type = 2024-09-13T23:43:52.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:52.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.939+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33800 con 0x55e1a6b6b400 2024-09-13T23:43:52.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.943+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.943+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94427652152064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33c00 con 0x55e1a6b6b400 2024-09-13T23:43:52.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.943+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c32c00 con 0x55e1a6b6b000 2024-09-13T23:43:52.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.947+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94427652152064] v35'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.947+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.947+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94427652153216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33000 con 0x55e1a6b6b000 2024-09-13T23:43:52.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.947+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33800 con 0x55e1a6b6b400 2024-09-13T23:43:52.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.947+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94427652153216] v35'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.947+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.947+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94427652154368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6bc00 con 0x55e1a6b6b400 2024-09-13T23:43:52.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.947+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6c32400 2024-09-13T23:43:52.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.951+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:52.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.951+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94427652154368] v35'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.951+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94427652155520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c32800 con 0x55e1a6c32400 2024-09-13T23:43:52.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.951+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33400 con 0x55e1a6b6b400 2024-09-13T23:43:52.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.955+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94427652155520] v35'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:52.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.955+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.955+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94427652156672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c32c00 con 0x55e1a6b6b400 2024-09-13T23:43:52.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.955+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33000 con 0x55e1a6c32000 2024-09-13T23:43:52.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.955+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.959+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94427652157824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c38000 con 0x55e1a6c32000 2024-09-13T23:43:52.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.959+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c38400 con 0x55e1a6c32000 2024-09-13T23:43:52.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.959+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94427652156672] v35'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.959+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v35'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.959+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94427652158976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c38800 con 0x55e1a6c32000 2024-09-13T23:43:52.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.959+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c38c00 con 0x55e1a6c32400 2024-09-13T23:43:52.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.959+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94427652157824] v35'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94427652158976] v35'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:52.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'180 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:52.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94427652160128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c39000 con 0x55e1a6c32400 2024-09-13T23:43:52.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c87419f40 20 add_watcher() i=0 2024-09-13T23:43:52.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c87419f40 20 add_watcher() i=1 2024-09-13T23:43:52.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c87419f40 20 add_watcher() i=2 2024-09-13T23:43:52.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c87419f40 20 add_watcher() i=3 2024-09-13T23:43:52.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c87419f40 20 add_watcher() i=4 2024-09-13T23:43:52.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c87419f40 20 add_watcher() i=5 2024-09-13T23:43:52.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c87419f40 20 add_watcher() i=6 2024-09-13T23:43:52.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94427652160128] v35'181 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:52.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c87419f40 20 add_watcher() i=7 2024-09-13T23:43:52.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c87419f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:52.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c39400 con 0x55e1a6b6b000 2024-09-13T23:43:52.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.963+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94427653857280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c39800 con 0x55e1a6b6b000 2024-09-13T23:43:52.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94427653857280] v35'187 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c6cba0700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:52.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c87419f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:52.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c87419f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:52.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c87419f40 1 Processor -- start 2024-09-13T23:43:52.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c87419f40 1 -- start start 2024-09-13T23:43:52.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c87419f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 0x55e1a6aa9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c87419f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a684e3c0 con 0x55e1a6c38c00 2024-09-13T23:43:52.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c853d1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 0x55e1a6aa9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c853d1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 0x55e1a6aa9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52548/0 (socket says 172.21.15.167:52548) 2024-09-13T23:43:52.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c853d1700 1 -- 172.21.15.167:0/327046624 learned_addr learned my addr 172.21.15.167:0/327046624 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c853d1700 1 -- 172.21.15.167:0/327046624 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1a67d2b60 con 0x55e1a6c38c00 2024-09-13T23:43:52.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c853d1700 1 --2- 172.21.15.167:0/327046624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 0x55e1a6aa9180 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55e1a6c46090 tx=0x55e1a6c14060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd188bbc389ca925 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c6c39f700 1 -- 172.21.15.167:0/327046624 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a6c4e000 con 0x55e1a6c38c00 2024-09-13T23:43:52.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c6c39f700 1 -- 172.21.15.167:0/327046624 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1a6b7a600 con 0x55e1a6c38c00 2024-09-13T23:43:52.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.967+0000 7f0c6c39f700 1 -- 172.21.15.167:0/327046624 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a6c4e160 con 0x55e1a6c38c00 2024-09-13T23:43:52.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 -- 172.21.15.167:0/327046624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 msgr2=0x55e1a6aa9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:52.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 --2- 172.21.15.167:0/327046624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 0x55e1a6aa9180 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x55e1a6c46090 tx=0x55e1a6c14060 comp rx=0 tx=0).stop 2024-09-13T23:43:52.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 -- 172.21.15.167:0/327046624 shutdown_connections 2024-09-13T23:43:52.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 --2- 172.21.15.167:0/327046624 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 0x55e1a6aa9180 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:52.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 -- 172.21.15.167:0/327046624 >> 172.21.15.167:0/327046624 conn(0x55e1a6c39c00 msgr2=0x55e1a68c2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:52.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 -- 172.21.15.167:0/327046624 shutdown_connections 2024-09-13T23:43:52.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 -- 172.21.15.167:0/327046624 wait complete. 2024-09-13T23:43:52.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 Processor -- start 2024-09-13T23:43:52.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 -- start start 2024-09-13T23:43:52.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 0x55e1a6aa9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1a684e500 con 0x55e1a6c38c00 2024-09-13T23:43:52.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c853d1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 0x55e1a6aa9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c853d1700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 0x55e1a6aa9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52550/0 (socket says 172.21.15.167:52550) 2024-09-13T23:43:52.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c853d1700 1 -- 172.21.15.167:0/2505439017 learned_addr learned my addr 172.21.15.167:0/2505439017 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:52.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c853d1700 1 -- 172.21.15.167:0/2505439017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1a67d2d00 con 0x55e1a6c38c00 2024-09-13T23:43:52.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c853d1700 1 --2- 172.21.15.167:0/2505439017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 0x55e1a6aa9180 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x55e1a6c47350 tx=0x55e1a6c14180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c6ab9c700 1 -- 172.21.15.167:0/2505439017 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a6c4e2c0 con 0x55e1a6c38c00 2024-09-13T23:43:52.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c6ab9c700 1 -- 172.21.15.167:0/2505439017 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1a6b7a300 con 0x55e1a6c38c00 2024-09-13T23:43:52.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 -- 172.21.15.167:0/2505439017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1a67d2340 con 0x55e1a6c38c00 2024-09-13T23:43:52.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c6ab9c700 1 -- 172.21.15.167:0/2505439017 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e1a6c4e000 con 0x55e1a6c38c00 2024-09-13T23:43:52.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.971+0000 7f0c87419f40 1 -- 172.21.15.167:0/2505439017 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1a67d2b60 con 0x55e1a6c38c00 2024-09-13T23:43:52.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.975+0000 7f0c6ab9c700 1 -- 172.21.15.167:0/2505439017 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e1a68bbb80 con 0x55e1a6c38c00 2024-09-13T23:43:52.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.975+0000 7f0c6ab9c700 1 --2- 172.21.15.167:0/2505439017 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6ccc000 0x55e1a6cd2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:52.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.975+0000 7f0c6ab9c700 1 -- 172.21.15.167:0/2505439017 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55e1a6827500 con 0x55e1a6c38c00 2024-09-13T23:43:52.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.975+0000 7f0c87419f40 5 note: GC not initialized 2024-09-13T23:43:52.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.975+0000 7f0c863d3700 1 --2- 172.21.15.167:0/2505439017 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6ccc000 0x55e1a6cd2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:52.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.975+0000 7f0c863d3700 1 --2- 172.21.15.167:0/2505439017 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6ccc000 0x55e1a6cd2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55e1a6af3140 tx=0x55e1a6b2f500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:52.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.975+0000 7f0c68397700 20 reqs_thread_entry: start 2024-09-13T23:43:52.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.975+0000 7f0c87419f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:52.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:52.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c66393700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:23 2024 2024-09-13T23:43:52.987 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:52.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c65b71700 20 reqs_thread_entry: start 2024-09-13T23:43:52.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:52.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:52.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c38000 con 0x55e1a6b6b000 2024-09-13T23:43:52.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv13 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:52.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:52.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:52.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:52.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~369 in=369b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c32800 con 0x55e1a6b6b000 2024-09-13T23:43:52.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~369] v35'14 uv14 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:52.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:52.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:52.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.979+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94427653859584 in=557b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33000 con 0x55e1a6b6b400 2024-09-13T23:43:52.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.983+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94427652156672 notify 150323855398 ret 0) v3 ==== 587+0+0 (crc 0 0 0) 0x55e1a6b22680 con 0x55e1a6b6b400 2024-09-13T23:43:52.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.983+0000 7f0c803c7700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855398 cookie 94427652156672 notifier 4710 bl.length()=545 2024-09-13T23:43:52.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.983+0000 7f0c803c7700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:52.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.983+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(64 notify.4 [notify cookie 94427653859584 out=8b] v0'0 uv180 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.983+0000 7f0c803c7700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:52.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.983+0000 7f0c803c7700 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33c00 con 0x55e1a6b6b400 2024-09-13T23:43:52.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.983+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.983+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== watch-notify(notify_complete (2) cookie 94427653859584 notify 150323855398 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e1a6b22820 con 0x55e1a6b6b400 2024-09-13T23:43:52.992 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:52.992 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:52.992 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:52.992 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:52.992 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:52.992 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:52.992 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:52.992 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:52.992 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [ 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "user", 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: "perm": "read" 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:52.993 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:52.994 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:52.995 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:52.995 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:52.995 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:52.995 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:52.995 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:52.995 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:52.995 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:52.995 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:52.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.983+0000 7f0c6cba0700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:52.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.983+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94427653857280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6b6b800 con 0x55e1a6b6b000 2024-09-13T23:43:52.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.987+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94427653857280] v35'188 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:52.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.987+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94427652152064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6ccc400 con 0x55e1a6b6b400 2024-09-13T23:43:52.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.987+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94427652152064] v35'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:52.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.991+0000 7f0c87419f40 20 remove_watcher() i=0 2024-09-13T23:43:52.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.991+0000 7f0c87419f40 2 removed watcher, disabling cache 2024-09-13T23:43:52.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.991+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94427652153216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6ccc800 con 0x55e1a6b6b000 2024-09-13T23:43:53.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.991+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94427652153216] v35'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6b6b000 2024-09-13T23:43:53.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.991+0000 7f0c87419f40 20 remove_watcher() i=1 2024-09-13T23:43:53.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.991+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94427652154368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6cccc00 con 0x55e1a6b6b400 2024-09-13T23:43:53.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.995+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94427652154368] v35'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:53.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.995+0000 7f0c87419f40 20 remove_watcher() i=2 2024-09-13T23:43:53.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.995+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94427652155520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6ccd000 con 0x55e1a6c32400 2024-09-13T23:43:53.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.995+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94427652155520] v35'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:53.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.995+0000 7f0c87419f40 20 remove_watcher() i=3 2024-09-13T23:43:53.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.995+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94427652156672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6ccd400 con 0x55e1a6b6b400 2024-09-13T23:43:53.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.999+0000 7f0c85bd2700 1 -- 172.21.15.167:0/3391884987 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94427652156672] v35'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f200 con 0x55e1a6b6b400 2024-09-13T23:43:53.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.999+0000 7f0c87419f40 20 remove_watcher() i=4 2024-09-13T23:43:53.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.999+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94427652157824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6ccd800 con 0x55e1a6c32000 2024-09-13T23:43:53.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.999+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94427652157824] v35'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:53.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.999+0000 7f0c87419f40 20 remove_watcher() i=5 2024-09-13T23:43:53.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:52.999+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94427652158976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6ccdc00 con 0x55e1a6c32000 2024-09-13T23:43:53.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.003+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94427652158976] v35'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2f440 con 0x55e1a6c32000 2024-09-13T23:43:53.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.003+0000 7f0c87419f40 20 remove_watcher() i=6 2024-09-13T23:43:53.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.003+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94427652160128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e1a6c33800 con 0x55e1a6c32400 2024-09-13T23:43:53.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c863d3700 1 -- 172.21.15.167:0/3391884987 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94427652160128] v35'182 uv180 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e1a6c2eb40 con 0x55e1a6c32400 2024-09-13T23:43:53.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 20 remove_watcher() i=7 2024-09-13T23:43:53.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e1a6b6b400 msgr2=0x55e1a6aa8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e1a6b6b400 0x55e1a6aa8100 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e1a6b6b000 msgr2=0x55e1a6aa7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e1a6b6b000 0x55e1a6aa7b80 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e1a6c32400 msgr2=0x55e1a6aa8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e1a6c32400 0x55e1a6aa8c00 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e1a6c32000 msgr2=0x55e1a6aa8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e1a6c32000 0x55e1a6aa8680 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6b6ac00 msgr2=0x55e1a6aa7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6b6ac00 0x55e1a6aa7600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55e1a6b6de60 tx=0x55e1a6c140c0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 msgr2=0x55e1a6aa6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 0x55e1a6aa6b00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x55e1a6af21e0 tx=0x55e1a6b2ea20 comp rx=0 tx=0).stop 2024-09-13T23:43:53.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c853d1700 1 -- 172.21.15.167:0/3391884987 reap_dead start 2024-09-13T23:43:53.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 shutdown_connections 2024-09-13T23:43:53.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/3391884987 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1800 0x55e1a6aa6b00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 >> 172.21.15.167:0/3391884987 conn(0x55e1a67c1400 msgr2=0x55e1a68c1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 shutdown_connections 2024-09-13T23:43:53.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/3391884987 wait complete. 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/2505439017 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6ccc000 msgr2=0x55e1a6cd2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/2505439017 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6ccc000 0x55e1a6cd2000 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55e1a6af3140 tx=0x55e1a6b2f500 comp rx=0 tx=0).stop 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/2505439017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 msgr2=0x55e1a6aa9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/2505439017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 0x55e1a6aa9180 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x55e1a6c47350 tx=0x55e1a6c14180 comp rx=0 tx=0).stop 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/2505439017 shutdown_connections 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/2505439017 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6ccc000 0x55e1a6cd2000 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/2505439017 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a6c38c00 0x55e1a6aa9180 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/2505439017 >> 172.21.15.167:0/2505439017 conn(0x55e1a6c39c00 msgr2=0x55e1a68c2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/2505439017 shutdown_connections 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/2505439017 wait complete. 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/2441703037 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6b6a000 msgr2=0x55e1a6aa7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/2441703037 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6b6a000 0x55e1a6aa7080 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x55e1a6af2f90 tx=0x55e1a6b2eb40 comp rx=0 tx=0).stop 2024-09-13T23:43:53.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/2441703037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 msgr2=0x55e1a6aa6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/2441703037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x55e1a762f6b0 tx=0x55e1a68c5920 comp rx=0 tx=0).stop 2024-09-13T23:43:53.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/2441703037 shutdown_connections 2024-09-13T23:43:53.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/2441703037 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e1a6b6a000 0x55e1a6aa7080 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 --2- 172.21.15.167:0/2441703037 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e1a67c1000 0x55e1a6aa6580 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/2441703037 >> 172.21.15.167:0/2441703037 conn(0x55e1a67c0c00 msgr2=0x55e1a68c1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/2441703037 shutdown_connections 2024-09-13T23:43:53.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.007+0000 7f0c87419f40 1 -- 172.21.15.167:0/2441703037 wait complete. 2024-09-13T23:43:53.023 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [{'type': 'user', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:53.023 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-09-13T23:43:53.023 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-09-13T23:43:53.023 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid foo --caps user=read 2024-09-13T23:43:53.066 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:53.066 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:53.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d7255f40 1 Processor -- start 2024-09-13T23:43:53.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d7255f40 1 -- start start 2024-09-13T23:43:53.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d7255f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5590fa373400 0x5590fa658580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d7255f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5590fa437500 con 0x5590fa373000 2024-09-13T23:43:53.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d7255f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5590fa400a00 con 0x5590fa373400 2024-09-13T23:43:53.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d520d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5590fa373400 0x5590fa658580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d5a0e700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5590fa373000 0x5590fa472800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:49372/0 (socket says 172.21.15.167:49372) 2024-09-13T23:43:53.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d520d700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5590fa373400 0x5590fa658580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52560/0 (socket says 172.21.15.167:52560) 2024-09-13T23:43:53.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d5a0e700 1 -- 172.21.15.167:0/1293500120 learned_addr learned my addr 172.21.15.167:0/1293500120 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:53.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d4a0c700 1 -- 172.21.15.167:0/1293500120 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152996448 0 0) 0x5590fa4ce000 con 0x5590fa373000 2024-09-13T23:43:53.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d4a0c700 1 -- 172.21.15.167:0/1293500120 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x5590fa437680 con 0x5590fa373000 2024-09-13T23:43:53.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d520d700 1 -- 172.21.15.167:0/1293500120 >> v1:172.21.15.12:6789/0 conn(0x5590fa373000 legacy=0x5590fa472800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d520d700 1 -- 172.21.15.167:0/1293500120 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590fa385d40 con 0x5590fa373400 2024-09-13T23:43:53.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d520d700 1 --2- 172.21.15.167:0/1293500120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373400 0x5590fa658580 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x5590fa653920 tx=0x5590fa477980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f149b01c74a292 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d4a0c700 1 -- 172.21.15.167:0/1293500120 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa4174a0 con 0x5590fa373400 2024-09-13T23:43:53.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d4a0c700 1 -- 172.21.15.167:0/1293500120 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590fa437500 con 0x5590fa373400 2024-09-13T23:43:53.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.059+0000 7f63d4a0c700 1 -- 172.21.15.167:0/1293500120 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa417600 con 0x5590fa373400 2024-09-13T23:43:53.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 -- 172.21.15.167:0/1293500120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373400 msgr2=0x5590fa658580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 --2- 172.21.15.167:0/1293500120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373400 0x5590fa658580 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x5590fa653920 tx=0x5590fa477980 comp rx=0 tx=0).stop 2024-09-13T23:43:53.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 -- 172.21.15.167:0/1293500120 shutdown_connections 2024-09-13T23:43:53.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 --2- 172.21.15.167:0/1293500120 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373400 0x5590fa658580 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 -- 172.21.15.167:0/1293500120 >> 172.21.15.167:0/1293500120 conn(0x5590fa372c00 msgr2=0x5590fa472000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 -- 172.21.15.167:0/1293500120 shutdown_connections 2024-09-13T23:43:53.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 -- 172.21.15.167:0/1293500120 wait complete. 2024-09-13T23:43:53.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 Processor -- start 2024-09-13T23:43:53.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 -- start start 2024-09-13T23:43:53.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 0x5590fa658580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fa400500 con 0x5590fa373000 2024-09-13T23:43:53.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d5a0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 0x5590fa658580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d5a0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 0x5590fa658580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52572/0 (socket says 172.21.15.167:52572) 2024-09-13T23:43:53.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3527621757 learned_addr learned my addr 172.21.15.167:0/3527621757 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:53.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3527621757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590fa385ba0 con 0x5590fa373000 2024-09-13T23:43:53.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d5a0e700 1 --2- 172.21.15.167:0/3527621757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 0x5590fa658580 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x5590fb13dd70 tx=0x5590fa477b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c52a05277edf4a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d420b700 1 -- 172.21.15.167:0/3527621757 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa4174a0 con 0x5590fa373000 2024-09-13T23:43:53.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d420b700 1 -- 172.21.15.167:0/3527621757 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590fa437b00 con 0x5590fa373000 2024-09-13T23:43:53.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d420b700 1 -- 172.21.15.167:0/3527621757 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa417600 con 0x5590fa373000 2024-09-13T23:43:53.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 -- 172.21.15.167:0/3527621757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 msgr2=0x5590fa658580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3527621757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 0x5590fa658580 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x5590fb13dd70 tx=0x5590fa477b60 comp rx=0 tx=0).stop 2024-09-13T23:43:53.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 -- 172.21.15.167:0/3527621757 shutdown_connections 2024-09-13T23:43:53.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3527621757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 0x5590fa658580 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.063+0000 7f63d7255f40 1 -- 172.21.15.167:0/3527621757 >> 172.21.15.167:0/3527621757 conn(0x5590fa372c00 msgr2=0x5590fa473000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d7255f40 1 -- 172.21.15.167:0/3527621757 shutdown_connections 2024-09-13T23:43:53.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d7255f40 1 -- 172.21.15.167:0/3527621757 wait complete. 2024-09-13T23:43:53.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d7255f40 1 Processor -- start 2024-09-13T23:43:53.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d7255f40 1 -- start start 2024-09-13T23:43:53.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d7255f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 0x5590fa658580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d7255f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fa4003c0 con 0x5590fa373000 2024-09-13T23:43:53.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d5a0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 0x5590fa658580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d5a0e700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 0x5590fa658580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52578/0 (socket says 172.21.15.167:52578) 2024-09-13T23:43:53.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d5a0e700 1 -- 172.21.15.167:0/2701046781 learned_addr learned my addr 172.21.15.167:0/2701046781 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:53.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d5a0e700 1 -- 172.21.15.167:0/2701046781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590fa385a00 con 0x5590fa373000 2024-09-13T23:43:53.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d5a0e700 1 --2- 172.21.15.167:0/2701046781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 0x5590fa658580 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x5590fb13de60 tx=0x5590fa477920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d2a08700 1 -- 172.21.15.167:0/2701046781 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa4178c0 con 0x5590fa373000 2024-09-13T23:43:53.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d2a08700 1 -- 172.21.15.167:0/2701046781 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590fa437c80 con 0x5590fa373000 2024-09-13T23:43:53.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d7255f40 1 -- 172.21.15.167:0/2701046781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5590fa385860 con 0x5590fa373000 2024-09-13T23:43:53.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d2a08700 1 -- 172.21.15.167:0/2701046781 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa4174a0 con 0x5590fa373000 2024-09-13T23:43:53.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.067+0000 7f63d7255f40 1 -- 172.21.15.167:0/2701046781 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590fa3856c0 con 0x5590fa373000 2024-09-13T23:43:53.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d7255f40 1 Processor -- start 2024-09-13T23:43:53.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d7255f40 1 -- start start 2024-09-13T23:43:53.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d2a08700 1 -- 172.21.15.167:0/2701046781 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5590fa46cb00 con 0x5590fa373000 2024-09-13T23:43:53.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d7255f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 0x5590fa658b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d7255f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fa398640 con 0x5590fa373800 2024-09-13T23:43:53.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d2a08700 1 --2- 172.21.15.167:0/2701046781 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa712000 0x5590fa659080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d620f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 0x5590fa658b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d620f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 0x5590fa658b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52584/0 (socket says 172.21.15.167:52584) 2024-09-13T23:43:53.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d2a08700 1 -- 172.21.15.167:0/2701046781 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5590fa3d9180 con 0x5590fa373000 2024-09-13T23:43:53.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d620f700 1 -- 172.21.15.167:0/3672522835 learned_addr learned my addr 172.21.15.167:0/3672522835 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:53.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d5a0e700 1 --2- 172.21.15.167:0/2701046781 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa712000 0x5590fa659080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d620f700 1 -- 172.21.15.167:0/3672522835 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590fa384000 con 0x5590fa373800 2024-09-13T23:43:53.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d620f700 1 --2- 172.21.15.167:0/3672522835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 0x5590fa658b00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x5590fa6a1080 tx=0x5590fa6cede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e305540d5990adc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d5a0e700 1 --2- 172.21.15.167:0/2701046781 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa712000 0x5590fa659080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5590fa6a14d0 tx=0x5590fa6ce240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d0203700 1 -- 172.21.15.167:0/3672522835 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa4178c0 con 0x5590fa373800 2024-09-13T23:43:53.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d0203700 1 -- 172.21.15.167:0/3672522835 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590fb1dc000 con 0x5590fa373800 2024-09-13T23:43:53.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d0203700 1 -- 172.21.15.167:0/3672522835 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa4174a0 con 0x5590fa373800 2024-09-13T23:43:53.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d7255f40 1 -- 172.21.15.167:0/3672522835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 msgr2=0x5590fa658b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3672522835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 0x5590fa658b00 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x5590fa6a1080 tx=0x5590fa6cede0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d7255f40 1 -- 172.21.15.167:0/3672522835 shutdown_connections 2024-09-13T23:43:53.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3672522835 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 0x5590fa658b00 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.071+0000 7f63d7255f40 1 -- 172.21.15.167:0/3672522835 >> 172.21.15.167:0/3672522835 conn(0x5590fa373400 msgr2=0x5590fa473800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d7255f40 1 -- 172.21.15.167:0/3672522835 shutdown_connections 2024-09-13T23:43:53.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d7255f40 1 -- 172.21.15.167:0/3672522835 wait complete. 2024-09-13T23:43:53.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d7255f40 1 Processor -- start 2024-09-13T23:43:53.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d7255f40 1 -- start start 2024-09-13T23:43:53.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d7255f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 0x5590fa658b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d7255f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fa3988c0 con 0x5590fa373800 2024-09-13T23:43:53.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d620f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 0x5590fa658b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d620f700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 0x5590fa658b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52600/0 (socket says 172.21.15.167:52600) 2024-09-13T23:43:53.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 learned_addr learned my addr 172.21.15.167:0/3723767239 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:53.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590fa384ea0 con 0x5590fa373800 2024-09-13T23:43:53.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d620f700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 0x5590fa658b00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x5590fa6a00f0 tx=0x5590fa6ce840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63cea00700 1 -- 172.21.15.167:0/3723767239 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa417b80 con 0x5590fa373800 2024-09-13T23:43:53.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63cea00700 1 -- 172.21.15.167:0/3723767239 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590fb1dc180 con 0x5590fa373800 2024-09-13T23:43:53.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5590fa384820 con 0x5590fa373800 2024-09-13T23:43:53.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63cea00700 1 -- 172.21.15.167:0/3723767239 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa4178c0 con 0x5590fa373800 2024-09-13T23:43:53.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.075+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590fa3849c0 con 0x5590fa373800 2024-09-13T23:43:53.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63cea00700 1 -- 172.21.15.167:0/3723767239 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5590fa46d080 con 0x5590fa373800 2024-09-13T23:43:53.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63cea00700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa712c00 0x5590fa659600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63cea00700 1 -- 172.21.15.167:0/3723767239 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5590fa3d9340 con 0x5590fa373800 2024-09-13T23:43:53.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d520d700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa712c00 0x5590fa659600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5590fa713000 0x5590fa659b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713400 con 0x5590fa713000 2024-09-13T23:43:53.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d620f700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5590fa713000 0x5590fa659b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d520d700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa712c00 0x5590fa659600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5590fa653920 tx=0x5590fa477800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d620f700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5590fa713000 0x5590fa659b80 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 20 realm 2024-09-13T23:43:53.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713000 2024-09-13T23:43:53.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:53.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713c00 con 0x5590fa713000 2024-09-13T23:43:53.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5590fa713400 0x5590fa65a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713400 2024-09-13T23:43:53.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.079+0000 7f63d5a0e700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5590fa713400 0x5590fa65a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d5a0e700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5590fa713400 0x5590fa65a100 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d7255f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:53.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d7255f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713c00 con 0x5590fa713400 2024-09-13T23:43:53.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d7255f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:53.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d7255f40 20 searching for the correct realm 2024-09-13T23:43:53.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5590fa7de000 0x5590fa65a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7de400 con 0x5590fa7de000 2024-09-13T23:43:53.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d520d700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5590fa7de000 0x5590fa65a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d520d700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5590fa7de000 0x5590fa65a680 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7de400 con 0x5590fa713400 2024-09-13T23:43:53.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d5a0e700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5590fa7de400 0x5590fa65ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de400 2024-09-13T23:43:53.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.083+0000 7f63d620f700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5590fa7de400 0x5590fa65ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d620f700 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5590fa7de400 0x5590fa65ac00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713000 2024-09-13T23:43:53.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713000 2024-09-13T23:43:53.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de000 2024-09-13T23:43:53.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713000 2024-09-13T23:43:53.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713400 2024-09-13T23:43:53.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713400 2024-09-13T23:43:53.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713000 2024-09-13T23:43:53.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.087+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de400 2024-09-13T23:43:53.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713000 2024-09-13T23:43:53.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de400 2024-09-13T23:43:53.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713400 2024-09-13T23:43:53.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de000 2024-09-13T23:43:53.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713000 2024-09-13T23:43:53.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713400 2024-09-13T23:43:53.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de000 2024-09-13T23:43:53.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.091+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713000 2024-09-13T23:43:53.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de400 2024-09-13T23:43:53.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713000 2024-09-13T23:43:53.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de000 2024-09-13T23:43:53.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de400 2024-09-13T23:43:53.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713000 2024-09-13T23:43:53.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de400 2024-09-13T23:43:53.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de000 2024-09-13T23:43:53.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de400 2024-09-13T23:43:53.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.095+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713400 2024-09-13T23:43:53.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713400 2024-09-13T23:43:53.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de000 2024-09-13T23:43:53.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713400 2024-09-13T23:43:53.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713400 2024-09-13T23:43:53.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d7255f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:53.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d7255f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:53.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d7255f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:53.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d7255f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:53.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d7255f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7de800 con 0x5590fa713000 2024-09-13T23:43:53.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d7255f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d7255f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.099+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7dec00 con 0x5590fa713400 2024-09-13T23:43:53.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:53.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7df000 con 0x5590fa713400 2024-09-13T23:43:53.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:53.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 zone default found 2024-09-13T23:43:53.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 4 Realm: () 2024-09-13T23:43:53.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:53.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:53.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:53.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 zonegroup default 2024-09-13T23:43:53.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7df400 con 0x5590fa7de400 2024-09-13T23:43:53.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7df800 con 0x5590fa7de000 2024-09-13T23:43:53.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7dfc00 con 0x5590fa7de000 2024-09-13T23:43:53.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 started sync module instance, tier type = 2024-09-13T23:43:53.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:53.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.103+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7df800 con 0x5590fa713400 2024-09-13T23:43:53.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.107+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.107+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94081164573440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7dfc00 con 0x5590fa713400 2024-09-13T23:43:53.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.107+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7dec00 con 0x5590fa713000 2024-09-13T23:43:53.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.111+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94081164573440] v35'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.111+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.111+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94081164574592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7df000 con 0x5590fa713000 2024-09-13T23:43:53.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.111+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7df800 con 0x5590fa713400 2024-09-13T23:43:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.111+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94081164574592] v35'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.111+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.111+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94081164575744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713c00 con 0x5590fa713400 2024-09-13T23:43:53.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.111+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa7de400 2024-09-13T23:43:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.115+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.115+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94081164576896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7de800 con 0x5590fa7de400 2024-09-13T23:43:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.115+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7df400 con 0x5590fa713400 2024-09-13T23:43:53.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.115+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94081164575744] v35'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.119+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94081164576896] v35'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.119+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.119+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94081164578048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7dec00 con 0x5590fa713400 2024-09-13T23:43:53.127 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.119+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7df000 con 0x5590fa7de000 2024-09-13T23:43:53.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.119+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.119+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94081164579200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7e8000 con 0x5590fa7de000 2024-09-13T23:43:53.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.119+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7e8400 con 0x5590fa7de000 2024-09-13T23:43:53.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.119+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94081164578048] v35'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94081164579200] v35'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94081164580352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7e8800 con 0x5590fa7de000 2024-09-13T23:43:53.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7e8c00 con 0x5590fa7de400 2024-09-13T23:43:53.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94081164580352] v35'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'183 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94081164581504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7e9000 con 0x5590fa7de400 2024-09-13T23:43:53.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d7255f40 20 add_watcher() i=0 2024-09-13T23:43:53.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d7255f40 20 add_watcher() i=1 2024-09-13T23:43:53.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d7255f40 20 add_watcher() i=2 2024-09-13T23:43:53.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d7255f40 20 add_watcher() i=3 2024-09-13T23:43:53.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d7255f40 20 add_watcher() i=4 2024-09-13T23:43:53.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d7255f40 20 add_watcher() i=5 2024-09-13T23:43:53.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.123+0000 7f63d7255f40 20 add_watcher() i=6 2024-09-13T23:43:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.127+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94081164581504] v35'184 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.127+0000 7f63d7255f40 20 add_watcher() i=7 2024-09-13T23:43:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.127+0000 7f63d7255f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:53.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.127+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7e9400 con 0x5590fa713000 2024-09-13T23:43:53.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.127+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.136 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.127+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94081166270464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7e9800 con 0x5590fa713000 2024-09-13T23:43:53.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.127+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94081166270464] v35'189 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.127+0000 7f63bc9dc700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:53.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.127+0000 7f63d7255f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:53.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.127+0000 7f63d7255f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:53.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.127+0000 7f63d7255f40 1 Processor -- start 2024-09-13T23:43:53.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d7255f40 1 -- start start 2024-09-13T23:43:53.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d7255f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 0x5590fa65b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d7255f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fa4003c0 con 0x5590fa7e8c00 2024-09-13T23:43:53.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d520d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 0x5590fa65b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d520d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 0x5590fa65b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52604/0 (socket says 172.21.15.167:52604) 2024-09-13T23:43:53.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d520d700 1 -- 172.21.15.167:0/294394964 learned_addr learned my addr 172.21.15.167:0/294394964 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:53.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d520d700 1 -- 172.21.15.167:0/294394964 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590fa384b60 con 0x5590fa7e8c00 2024-09-13T23:43:53.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d520d700 1 --2- 172.21.15.167:0/294394964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 0x5590fa65b180 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x5590fa6a04b0 tx=0x5590fa477860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fb4bdd90780ade server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.140 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63bc1db700 1 -- 172.21.15.167:0/294394964 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa417760 con 0x5590fa7e8c00 2024-09-13T23:43:53.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63bc1db700 1 -- 172.21.15.167:0/294394964 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590fb1dc600 con 0x5590fa7e8c00 2024-09-13T23:43:53.141 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63bc1db700 1 -- 172.21.15.167:0/294394964 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa800000 con 0x5590fa7e8c00 2024-09-13T23:43:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d7255f40 1 -- 172.21.15.167:0/294394964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 msgr2=0x5590fa65b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d7255f40 1 --2- 172.21.15.167:0/294394964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 0x5590fa65b180 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x5590fa6a04b0 tx=0x5590fa477860 comp rx=0 tx=0).stop 2024-09-13T23:43:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d7255f40 1 -- 172.21.15.167:0/294394964 shutdown_connections 2024-09-13T23:43:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d7255f40 1 --2- 172.21.15.167:0/294394964 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 0x5590fa65b180 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d7255f40 1 -- 172.21.15.167:0/294394964 >> 172.21.15.167:0/294394964 conn(0x5590fa7e9c00 msgr2=0x5590fa474000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d7255f40 1 -- 172.21.15.167:0/294394964 shutdown_connections 2024-09-13T23:43:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d7255f40 1 -- 172.21.15.167:0/294394964 wait complete. 2024-09-13T23:43:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d7255f40 1 Processor -- start 2024-09-13T23:43:53.142 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.131+0000 7f63d7255f40 1 -- start start 2024-09-13T23:43:53.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63d7255f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 0x5590fa65b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63d7255f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590fa400500 con 0x5590fa7e8c00 2024-09-13T23:43:53.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63d520d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 0x5590fa65b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63d520d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 0x5590fa65b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52608/0 (socket says 172.21.15.167:52608) 2024-09-13T23:43:53.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63d520d700 1 -- 172.21.15.167:0/2900525936 learned_addr learned my addr 172.21.15.167:0/2900525936 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:53.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63d520d700 1 -- 172.21.15.167:0/2900525936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590fa384d00 con 0x5590fa7e8c00 2024-09-13T23:43:53.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63d520d700 1 --2- 172.21.15.167:0/2900525936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 0x5590fa65b180 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x5590fa7f27e0 tx=0x5590fa7c4420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63ba9d8700 1 -- 172.21.15.167:0/2900525936 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa800160 con 0x5590fa7e8c00 2024-09-13T23:43:53.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63ba9d8700 1 -- 172.21.15.167:0/2900525936 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590fb1dc300 con 0x5590fa7e8c00 2024-09-13T23:43:53.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63d7255f40 1 -- 172.21.15.167:0/2900525936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5590fa384340 con 0x5590fa7e8c00 2024-09-13T23:43:53.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63ba9d8700 1 -- 172.21.15.167:0/2900525936 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5590fa8002c0 con 0x5590fa7e8c00 2024-09-13T23:43:53.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63d7255f40 1 -- 172.21.15.167:0/2900525936 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590fa385d40 con 0x5590fa7e8c00 2024-09-13T23:43:53.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63ba9d8700 1 -- 172.21.15.167:0/2900525936 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5590fa46db80 con 0x5590fa7e8c00 2024-09-13T23:43:53.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63ba9d8700 1 --2- 172.21.15.167:0/2900525936 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa872000 0x5590fa878000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63ba9d8700 1 -- 172.21.15.167:0/2900525936 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5590fa3d9500 con 0x5590fa7e8c00 2024-09-13T23:43:53.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63d7255f40 5 note: GC not initialized 2024-09-13T23:43:53.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.135+0000 7f63d620f700 1 --2- 172.21.15.167:0/2900525936 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa872000 0x5590fa878000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.146 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63d620f700 1 --2- 172.21.15.167:0/2900525936 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa872000 0x5590fa878000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5590fa76dd70 tx=0x5590fa6cf7a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63d7255f40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:53.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63b81d3700 20 reqs_thread_entry: start 2024-09-13T23:43:53.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63d7255f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:53.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63b59ce700 20 reqs_thread_entry: start 2024-09-13T23:43:53.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63d7255f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:53.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63d7255f40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7e8000 con 0x5590fa713000 2024-09-13T23:43:53.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63b61cf700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:23 2024 2024-09-13T23:43:53.149 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:53.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call out=48b,read 0~369 out=369b,getxattrs out=75b] v0'0 uv14 ondisk = 0) v8 ==== 231+0+492 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63d7255f40 20 rados_obj.operate() r=0 bl.length=369 2024-09-13T23:43:53.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63d7255f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:53.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63d7255f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:53.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63d7255f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:53.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.139+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~357 in=357b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7de800 con 0x5590fa713000 2024-09-13T23:43:53.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~357] v35'15 uv15 ondisk = 0) v8 ==== 315+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d7255f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:53.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d7255f40 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:53.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d7255f40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:43:53.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94081166272768 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7df000 con 0x5590fa713400 2024-09-13T23:43:53.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94081164578048 notify 150323855399 ret 0) v3 ==== 575+0+0 (crc 0 0 0) 0x5590fa6cc820 con 0x5590fa713400 2024-09-13T23:43:53.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d0203700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855399 cookie 94081164578048 notifier 4718 bl.length()=533 2024-09-13T23:43:53.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(64 notify.4 [notify cookie 94081166272768 out=8b] v0'0 uv183 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d0203700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-09-13T23:43:53.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d0203700 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:53.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d0203700 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7dfc00 con 0x5590fa713400 2024-09-13T23:43:53.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== watch-notify(notify_complete (2) cookie 94081166272768 notify 150323855399 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5590fa6cc9c0 con 0x5590fa713400 2024-09-13T23:43:53.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.143+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.154 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:43:53.154 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "foo", 2024-09-13T23:43:53.154 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Foo", 2024-09-13T23:43:53.154 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "foo@foo.com", 2024-09-13T23:43:53.154 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:43:53.154 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:43:53.154 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:43:53.154 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "9te6NH5mcdcq0Tc5i8i1", 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu" 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:43:53.155 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:43:53.156 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:43:53.157 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:43:53.157 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:43:53.157 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:43:53.157 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:43:53.157 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:43:53.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.147+0000 7f63bc9dc700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:53.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.147+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94081166270464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa713800 con 0x5590fa713000 2024-09-13T23:43:53.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.147+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94081166270464] v35'190 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.147+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94081164573440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa872400 con 0x5590fa713400 2024-09-13T23:43:53.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.151+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94081164573440] v35'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.151+0000 7f63d7255f40 20 remove_watcher() i=0 2024-09-13T23:43:53.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.151+0000 7f63d7255f40 2 removed watcher, disabling cache 2024-09-13T23:43:53.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.151+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94081164574592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa872800 con 0x5590fa713000 2024-09-13T23:43:53.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.155+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94081164574592] v35'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa713000 2024-09-13T23:43:53.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.155+0000 7f63d7255f40 20 remove_watcher() i=1 2024-09-13T23:43:53.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.155+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94081164575744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa872c00 con 0x5590fa713400 2024-09-13T23:43:53.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.155+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94081164575744] v35'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.155+0000 7f63d7255f40 20 remove_watcher() i=2 2024-09-13T23:43:53.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.155+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94081164576896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa873000 con 0x5590fa7de400 2024-09-13T23:43:53.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.159+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94081164576896] v35'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.159+0000 7f63d7255f40 20 remove_watcher() i=3 2024-09-13T23:43:53.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.159+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94081164578048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa873400 con 0x5590fa713400 2024-09-13T23:43:53.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.163+0000 7f63d5a0e700 1 -- 172.21.15.167:0/3723767239 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94081164578048] v35'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7200 con 0x5590fa713400 2024-09-13T23:43:53.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.163+0000 7f63d7255f40 20 remove_watcher() i=4 2024-09-13T23:43:53.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.163+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94081164579200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa873800 con 0x5590fa7de000 2024-09-13T23:43:53.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.163+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94081164579200] v35'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.163+0000 7f63d7255f40 20 remove_watcher() i=5 2024-09-13T23:43:53.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.163+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94081164580352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa873c00 con 0x5590fa7de000 2024-09-13T23:43:53.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94081164580352] v35'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b7440 con 0x5590fa7de000 2024-09-13T23:43:53.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 20 remove_watcher() i=6 2024-09-13T23:43:53.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94081164581504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5590fa7df800 con 0x5590fa7de400 2024-09-13T23:43:53.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d620f700 1 -- 172.21.15.167:0/3723767239 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94081164581504] v35'185 uv183 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5590fa7b6b40 con 0x5590fa7de400 2024-09-13T23:43:53.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 20 remove_watcher() i=7 2024-09-13T23:43:53.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5590fa713400 msgr2=0x5590fa65a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5590fa713400 0x5590fa65a100 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5590fa713000 msgr2=0x5590fa659b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5590fa713000 0x5590fa659b80 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5590fa7de400 msgr2=0x5590fa65ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5590fa7de400 0x5590fa65ac00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5590fa7de000 msgr2=0x5590fa65a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5590fa7de000 0x5590fa65a680 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa712c00 msgr2=0x5590fa659600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa712c00 0x5590fa659600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x5590fa653920 tx=0x5590fa477800 comp rx=0 tx=0).stop 2024-09-13T23:43:53.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 msgr2=0x5590fa658b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d7255f40 1 --2- 172.21.15.167:0/3723767239 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373800 0x5590fa658b00 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x5590fa6a00f0 tx=0x5590fa6ce840 comp rx=0 tx=0).stop 2024-09-13T23:43:53.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.167+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 reap_dead start 2024-09-13T23:43:53.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d520d700 1 -- 172.21.15.167:0/3723767239 reap_dead start 2024-09-13T23:43:53.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 shutdown_connections 2024-09-13T23:43:53.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 >> 172.21.15.167:0/3723767239 conn(0x5590fa373400 msgr2=0x5590fa473800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 shutdown_connections 2024-09-13T23:43:53.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/3723767239 wait complete. 2024-09-13T23:43:53.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/2900525936 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa872000 msgr2=0x5590fa878000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 --2- 172.21.15.167:0/2900525936 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa872000 0x5590fa878000 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x5590fa76dd70 tx=0x5590fa6cf7a0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/2900525936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 msgr2=0x5590fa65b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 --2- 172.21.15.167:0/2900525936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 0x5590fa65b180 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x5590fa7f27e0 tx=0x5590fa7c4420 comp rx=0 tx=0).stop 2024-09-13T23:43:53.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/2900525936 shutdown_connections 2024-09-13T23:43:53.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 --2- 172.21.15.167:0/2900525936 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa872000 0x5590fa878000 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 --2- 172.21.15.167:0/2900525936 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa7e8c00 0x5590fa65b180 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/2900525936 >> 172.21.15.167:0/2900525936 conn(0x5590fa7e9c00 msgr2=0x5590fa474000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/2900525936 shutdown_connections 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/2900525936 wait complete. 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/2701046781 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa712000 msgr2=0x5590fa659080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 --2- 172.21.15.167:0/2701046781 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa712000 0x5590fa659080 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x5590fa6a14d0 tx=0x5590fa6ce240 comp rx=0 tx=0).stop 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/2701046781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 msgr2=0x5590fa658580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 --2- 172.21.15.167:0/2701046781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 0x5590fa658580 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x5590fb13de60 tx=0x5590fa477920 comp rx=0 tx=0).stop 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/2701046781 shutdown_connections 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 --2- 172.21.15.167:0/2701046781 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5590fa712000 0x5590fa659080 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 --2- 172.21.15.167:0/2701046781 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5590fa373000 0x5590fa658580 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/2701046781 >> 172.21.15.167:0/2701046781 conn(0x5590fa372c00 msgr2=0x5590fa473000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/2701046781 shutdown_connections 2024-09-13T23:43:53.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.171+0000 7f63d7255f40 1 -- 172.21.15.167:0/2701046781 wait complete. 2024-09-13T23:43:53.186 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:43:53.206 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:53.206 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-09-13T23:43:53.206 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-09-13T23:43:53.207 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-09-13T23:43:53.247 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:43:53.248 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:43:53.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.239+0000 7fdfac25ff40 1 Processor -- start 2024-09-13T23:43:53.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.239+0000 7fdfac25ff40 1 -- start start 2024-09-13T23:43:53.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.239+0000 7fdfac25ff40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5650bf5eb000 0x5650bf8d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.239+0000 7fdfac25ff40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5650bf6af500 con 0x5650bf5eb400 2024-09-13T23:43:53.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.239+0000 7fdfac25ff40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5650bf678a00 con 0x5650bf5eb000 2024-09-13T23:43:53.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.239+0000 7fdfaaa18700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5650bf5eb000 0x5650bf8d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.239+0000 7fdfaa217700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5650bf5eb400 0x5650bf6ea800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:49382/0 (socket says 172.21.15.167:49382) 2024-09-13T23:43:53.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.239+0000 7fdfaaa18700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5650bf5eb000 0x5650bf8d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52620/0 (socket says 172.21.15.167:52620) 2024-09-13T23:43:53.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.239+0000 7fdfaa217700 1 -- 172.21.15.167:0/1365027929 learned_addr learned my addr 172.21.15.167:0/1365027929 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:43:53.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.239+0000 7fdfaaa18700 1 -- 172.21.15.167:0/1365027929 >> v1:172.21.15.12:6789/0 conn(0x5650bf5eb400 legacy=0x5650bf6ea800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.239+0000 7fdfaaa18700 1 -- 172.21.15.167:0/1365027929 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650bf5fdd40 con 0x5650bf5eb000 2024-09-13T23:43:53.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.239+0000 7fdfaaa18700 1 --2- 172.21.15.167:0/1365027929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x5650bf8cba70 tx=0x5650bf6ef920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6ddef9c0dc77392 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfa9a16700 1 -- 172.21.15.167:0/1365027929 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bf68f4a0 con 0x5650bf5eb000 2024-09-13T23:43:53.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfa9a16700 1 -- 172.21.15.167:0/1365027929 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650bf6af800 con 0x5650bf5eb000 2024-09-13T23:43:53.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfa9a16700 1 -- 172.21.15.167:0/1365027929 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bf68f600 con 0x5650bf5eb000 2024-09-13T23:43:53.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfac25ff40 1 -- 172.21.15.167:0/1365027929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 msgr2=0x5650bf8d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/1365027929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x5650bf8cba70 tx=0x5650bf6ef920 comp rx=0 tx=0).stop 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfac25ff40 1 -- 172.21.15.167:0/1365027929 shutdown_connections 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/1365027929 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfac25ff40 1 -- 172.21.15.167:0/1365027929 >> 172.21.15.167:0/1365027929 conn(0x5650bf5eac00 msgr2=0x5650bf6ea000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfac25ff40 1 -- 172.21.15.167:0/1365027929 shutdown_connections 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfac25ff40 1 -- 172.21.15.167:0/1365027929 wait complete. 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfac25ff40 1 Processor -- start 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfac25ff40 1 -- start start 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfac25ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfac25ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf678500 con 0x5650bf5eb000 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfaaa18700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfaaa18700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52624/0 (socket says 172.21.15.167:52624) 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3406023519 learned_addr learned my addr 172.21.15.167:0/3406023519 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:53.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3406023519 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650bf5fdba0 con 0x5650bf5eb000 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfaaa18700 1 --2- 172.21.15.167:0/3406023519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x5650c044f800 tx=0x5650bf6ef800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a6604f6562ea87 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfa9215700 1 -- 172.21.15.167:0/3406023519 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bf68f760 con 0x5650bf5eb000 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfa9215700 1 -- 172.21.15.167:0/3406023519 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650bf6afb00 con 0x5650bf5eb000 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.243+0000 7fdfa9215700 1 -- 172.21.15.167:0/3406023519 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bf68f4a0 con 0x5650bf5eb000 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3406023519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 msgr2=0x5650bf8d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3406023519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x5650c044f800 tx=0x5650bf6ef800 comp rx=0 tx=0).stop 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3406023519 shutdown_connections 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3406023519 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3406023519 >> 172.21.15.167:0/3406023519 conn(0x5650bf5eac00 msgr2=0x5650bf6eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3406023519 shutdown_connections 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3406023519 wait complete. 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 Processor -- start 2024-09-13T23:43:53.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 -- start start 2024-09-13T23:43:53.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf6783c0 con 0x5650bf5eb000 2024-09-13T23:43:53.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfaaa18700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfaaa18700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52628/0 (socket says 172.21.15.167:52628) 2024-09-13T23:43:53.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2600688409 learned_addr learned my addr 172.21.15.167:0/2600688409 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:53.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2600688409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650bf5fda00 con 0x5650bf5eb000 2024-09-13T23:43:53.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfaaa18700 1 --2- 172.21.15.167:0/2600688409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5650c044f6b0 tx=0x5650bf6ef920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfa7a12700 1 -- 172.21.15.167:0/2600688409 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bf68f600 con 0x5650bf5eb000 2024-09-13T23:43:53.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfa7a12700 1 -- 172.21.15.167:0/2600688409 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650bf6afc80 con 0x5650bf5eb000 2024-09-13T23:43:53.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2600688409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5650bf5fd860 con 0x5650bf5eb000 2024-09-13T23:43:53.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfa7a12700 1 -- 172.21.15.167:0/2600688409 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bf68f8c0 con 0x5650bf5eb000 2024-09-13T23:43:53.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.247+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2600688409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650bf5fd6c0 con 0x5650bf5eb000 2024-09-13T23:43:53.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfac25ff40 1 Processor -- start 2024-09-13T23:43:53.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfa7a12700 1 -- 172.21.15.167:0/2600688409 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5650bf6e4b00 con 0x5650bf5eb000 2024-09-13T23:43:53.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfac25ff40 1 -- start start 2024-09-13T23:43:53.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfac25ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 0x5650bf8d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfac25ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf610640 con 0x5650bf5eb800 2024-09-13T23:43:53.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfa7a12700 1 --2- 172.21.15.167:0/2600688409 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bf988000 0x5650bf8d1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfa7a12700 1 -- 172.21.15.167:0/2600688409 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5650bf651180 con 0x5650bf5eb000 2024-09-13T23:43:53.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfab219700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 0x5650bf8d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfab219700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 0x5650bf8d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52640/0 (socket says 172.21.15.167:52640) 2024-09-13T23:43:53.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfab219700 1 -- 172.21.15.167:0/3602836363 learned_addr learned my addr 172.21.15.167:0/3602836363 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:53.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfaaa18700 1 --2- 172.21.15.167:0/2600688409 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bf988000 0x5650bf8d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfab219700 1 -- 172.21.15.167:0/3602836363 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650bf5fc000 con 0x5650bf5eb800 2024-09-13T23:43:53.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfab219700 1 --2- 172.21.15.167:0/3602836363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 0x5650bf8d0b00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x5650c045e930 tx=0x5650bf94eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4bb8e8c385299a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfaaa18700 1 --2- 172.21.15.167:0/2600688409 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bf988000 0x5650bf8d1080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5650c045f080 tx=0x5650bf94ec60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfa520d700 1 -- 172.21.15.167:0/3602836363 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bf68f600 con 0x5650bf5eb800 2024-09-13T23:43:53.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfa520d700 1 -- 172.21.15.167:0/3602836363 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650bf99a000 con 0x5650bf5eb800 2024-09-13T23:43:53.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfa520d700 1 -- 172.21.15.167:0/3602836363 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bf68f8c0 con 0x5650bf5eb800 2024-09-13T23:43:53.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3602836363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 msgr2=0x5650bf8d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3602836363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 0x5650bf8d0b00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x5650c045e930 tx=0x5650bf94eb40 comp rx=0 tx=0).stop 2024-09-13T23:43:53.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3602836363 shutdown_connections 2024-09-13T23:43:53.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3602836363 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 0x5650bf8d0b00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3602836363 >> 172.21.15.167:0/3602836363 conn(0x5650bf5eb400 msgr2=0x5650bf6eb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3602836363 shutdown_connections 2024-09-13T23:43:53.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.251+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3602836363 wait complete. 2024-09-13T23:43:53.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfac25ff40 1 Processor -- start 2024-09-13T23:43:53.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfac25ff40 1 -- start start 2024-09-13T23:43:53.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfac25ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 0x5650bf8d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfac25ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf6108c0 con 0x5650bf5eb800 2024-09-13T23:43:53.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfab219700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 0x5650bf8d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfab219700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 0x5650bf8d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52656/0 (socket says 172.21.15.167:52656) 2024-09-13T23:43:53.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 learned_addr learned my addr 172.21.15.167:0/3465567726 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:53.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650bf5fcea0 con 0x5650bf5eb800 2024-09-13T23:43:53.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfab219700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 0x5650bf8d0b00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5650c045e2a0 tx=0x5650bf6efce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfa3a0a700 1 -- 172.21.15.167:0/3465567726 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bf68f760 con 0x5650bf5eb800 2024-09-13T23:43:53.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfa3a0a700 1 -- 172.21.15.167:0/3465567726 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650bf99a180 con 0x5650bf5eb800 2024-09-13T23:43:53.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5650bf5fc820 con 0x5650bf5eb800 2024-09-13T23:43:53.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfa3a0a700 1 -- 172.21.15.167:0/3465567726 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bf68f600 con 0x5650bf5eb800 2024-09-13T23:43:53.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650bf5fc9c0 con 0x5650bf5eb800 2024-09-13T23:43:53.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.255+0000 7fdfac25ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfa3a0a700 1 -- 172.21.15.167:0/3465567726 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5650bf6e4dc0 con 0x5650bf5eb800 2024-09-13T23:43:53.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfa3a0a700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bf988c00 0x5650bf8d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfa3a0a700 1 -- 172.21.15.167:0/3465567726 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5650bf651340 con 0x5650bf5eb800 2024-09-13T23:43:53.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfaa217700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bf988c00 0x5650bf8d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5650bf989000 0x5650bf8d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989400 con 0x5650bf989000 2024-09-13T23:43:53.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfab219700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5650bf989000 0x5650bf8d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfaa217700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bf988c00 0x5650bf8d1600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5650bf98bec0 tx=0x5650bfa3a0c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfab219700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5650bf989000 0x5650bf8d1b80 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 20 realm 2024-09-13T23:43:53.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989000 2024-09-13T23:43:53.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:43:53.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989c00 con 0x5650bf989000 2024-09-13T23:43:53.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5650bf989400 0x5650bf8d2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.259+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989400 2024-09-13T23:43:53.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfaaa18700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5650bf989400 0x5650bf8d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfaaa18700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5650bf989400 0x5650bf8d2100 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfac25ff40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:53.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfac25ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989c00 con 0x5650bf989400 2024-09-13T23:43:53.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfac25ff40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:43:53.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfac25ff40 20 searching for the correct realm 2024-09-13T23:43:53.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5650bfa52000 0x5650bf8d2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa52400 con 0x5650bfa52000 2024-09-13T23:43:53.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfaa217700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5650bfa52000 0x5650bf8d2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.263+0000 7fdfaa217700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5650bfa52000 0x5650bf8d2680 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa52400 con 0x5650bf989400 2024-09-13T23:43:53.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfaaa18700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5650bfa52400 0x5650bf8d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52400 2024-09-13T23:43:53.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfab219700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5650bfa52400 0x5650bf8d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfab219700 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5650bfa52400 0x5650bf8d2c00 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989000 2024-09-13T23:43:53.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989000 2024-09-13T23:43:53.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.267+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52000 2024-09-13T23:43:53.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.271+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.271+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989000 2024-09-13T23:43:53.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.271+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.271+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989400 2024-09-13T23:43:53.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.271+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.271+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989400 2024-09-13T23:43:53.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.271+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.271+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989000 2024-09-13T23:43:53.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.271+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.271+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52400 2024-09-13T23:43:53.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.271+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.271+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989000 2024-09-13T23:43:53.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.275+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.275+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52400 2024-09-13T23:43:53.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.275+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.275+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989400 2024-09-13T23:43:53.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.275+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.275+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52000 2024-09-13T23:43:53.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.275+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.275+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989000 2024-09-13T23:43:53.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.275+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.275+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989400 2024-09-13T23:43:53.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.275+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.275+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52000 2024-09-13T23:43:53.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989000 2024-09-13T23:43:53.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52400 2024-09-13T23:43:53.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989000 2024-09-13T23:43:53.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52000 2024-09-13T23:43:53.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52400 2024-09-13T23:43:53.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989000 2024-09-13T23:43:53.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.279+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52400 2024-09-13T23:43:53.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.283+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.283+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52000 2024-09-13T23:43:53.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.283+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.283+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52400 2024-09-13T23:43:53.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.283+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.283+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989400 2024-09-13T23:43:53.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.283+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.283+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989400 2024-09-13T23:43:53.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.283+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.283+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52000 2024-09-13T23:43:53.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.283+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.283+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989400 2024-09-13T23:43:53.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bf989400 2024-09-13T23:43:53.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfac25ff40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:43:53.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfac25ff40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:43:53.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfac25ff40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:43:53.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfac25ff40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:43:53.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfac25ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa52800 con 0x5650bf989000 2024-09-13T23:43:53.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfac25ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfac25ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa52c00 con 0x5650bf989400 2024-09-13T23:43:53.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfac25ff40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:43:53.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfac25ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.287+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53000 con 0x5650bf989400 2024-09-13T23:43:53.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:43:53.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 20 zone default found 2024-09-13T23:43:53.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 4 Realm: () 2024-09-13T23:43:53.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:43:53.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:43:53.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:43:53.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 20 zonegroup default 2024-09-13T23:43:53.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53400 con 0x5650bfa52400 2024-09-13T23:43:53.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53800 con 0x5650bfa52000 2024-09-13T23:43:53.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53c00 con 0x5650bfa52000 2024-09-13T23:43:53.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:43:53.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 20 started sync module instance, tier type = 2024-09-13T23:43:53.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:43:53.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.291+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53800 con 0x5650bf989400 2024-09-13T23:43:53.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.295+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.295+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94904811027200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53c00 con 0x5650bf989400 2024-09-13T23:43:53.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.295+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa52c00 con 0x5650bf989000 2024-09-13T23:43:53.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.299+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.299+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94904811028352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53000 con 0x5650bf989000 2024-09-13T23:43:53.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.299+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53800 con 0x5650bf989400 2024-09-13T23:43:53.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.299+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94904811027200] v35'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.299+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94904811028352] v35'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.299+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.299+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94904811029504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989c00 con 0x5650bf989400 2024-09-13T23:43:53.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.299+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989800 con 0x5650bfa52400 2024-09-13T23:43:53.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.303+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.303+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94904811030656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa52800 con 0x5650bfa52400 2024-09-13T23:43:53.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.303+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53400 con 0x5650bf989400 2024-09-13T23:43:53.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.303+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94904811029504] v35'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.307+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94904811030656] v35'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.307+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.307+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94904811031808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa52c00 con 0x5650bf989400 2024-09-13T23:43:53.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.307+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53000 con 0x5650bfa52000 2024-09-13T23:43:53.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.307+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.307+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94904811032960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa58000 con 0x5650bfa52000 2024-09-13T23:43:53.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.307+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa58400 con 0x5650bfa52000 2024-09-13T23:43:53.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.307+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94904811031808] v35'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v35'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94904811034112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa58800 con 0x5650bfa52000 2024-09-13T23:43:53.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa58c00 con 0x5650bfa52400 2024-09-13T23:43:53.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94904811032960] v35'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94904811034112] v35'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'186 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94904811035264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa59000 con 0x5650bfa52400 2024-09-13T23:43:53.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfac25ff40 20 add_watcher() i=0 2024-09-13T23:43:53.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfac25ff40 20 add_watcher() i=1 2024-09-13T23:43:53.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfac25ff40 20 add_watcher() i=2 2024-09-13T23:43:53.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfac25ff40 20 add_watcher() i=3 2024-09-13T23:43:53.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfac25ff40 20 add_watcher() i=4 2024-09-13T23:43:53.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfac25ff40 20 add_watcher() i=5 2024-09-13T23:43:53.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.311+0000 7fdfac25ff40 20 add_watcher() i=6 2024-09-13T23:43:53.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.315+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94904811035264] v35'187 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.315+0000 7fdfac25ff40 20 add_watcher() i=7 2024-09-13T23:43:53.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.315+0000 7fdfac25ff40 2 all 8 watchers are set, enabling cache 2024-09-13T23:43:53.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.315+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa59400 con 0x5650bf989000 2024-09-13T23:43:53.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.315+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.315+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94904812691456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa59800 con 0x5650bf989000 2024-09-13T23:43:53.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94904812691456] v35'191 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdf919e6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:53.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdfac25ff40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:43:53.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdfac25ff40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:43:53.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdfac25ff40 1 Processor -- start 2024-09-13T23:43:53.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdfac25ff40 1 -- start start 2024-09-13T23:43:53.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdfac25ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 0x5650bf8d3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdfac25ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf6783c0 con 0x5650bfa58c00 2024-09-13T23:43:53.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdfaa217700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 0x5650bf8d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdfaa217700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 0x5650bf8d3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52664/0 (socket says 172.21.15.167:52664) 2024-09-13T23:43:53.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdfaa217700 1 -- 172.21.15.167:0/3991868876 learned_addr learned my addr 172.21.15.167:0/3991868876 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:53.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdfaa217700 1 -- 172.21.15.167:0/3991868876 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650bf5fcb60 con 0x5650bfa58c00 2024-09-13T23:43:53.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdfaa217700 1 --2- 172.21.15.167:0/3991868876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 0x5650bf8d3180 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5650bfa68090 tx=0x5650bfa3a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60c9d55742bf953 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdf911e5700 1 -- 172.21.15.167:0/3991868876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bfa70000 con 0x5650bfa58c00 2024-09-13T23:43:53.330 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.319+0000 7fdf911e5700 1 -- 172.21.15.167:0/3991868876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650bf99a600 con 0x5650bfa58c00 2024-09-13T23:43:53.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdf911e5700 1 -- 172.21.15.167:0/3991868876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bfa70160 con 0x5650bfa58c00 2024-09-13T23:43:53.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3991868876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 msgr2=0x5650bf8d3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3991868876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 0x5650bf8d3180 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x5650bfa68090 tx=0x5650bfa3a060 comp rx=0 tx=0).stop 2024-09-13T23:43:53.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3991868876 shutdown_connections 2024-09-13T23:43:53.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3991868876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 0x5650bf8d3180 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3991868876 >> 172.21.15.167:0/3991868876 conn(0x5650bfa59c00 msgr2=0x5650bf6ec000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3991868876 shutdown_connections 2024-09-13T23:43:53.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3991868876 wait complete. 2024-09-13T23:43:53.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 Processor -- start 2024-09-13T23:43:53.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 -- start start 2024-09-13T23:43:53.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 0x5650bf8d3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650bf678500 con 0x5650bfa58c00 2024-09-13T23:43:53.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfaa217700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 0x5650bf8d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfaa217700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 0x5650bf8d3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:52680/0 (socket says 172.21.15.167:52680) 2024-09-13T23:43:53.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfaa217700 1 -- 172.21.15.167:0/2502811804 learned_addr learned my addr 172.21.15.167:0/2502811804 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:43:53.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfaa217700 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650bf5fcd00 con 0x5650bfa58c00 2024-09-13T23:43:53.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfaa217700 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 0x5650bf8d3180 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x5650bfa69350 tx=0x5650bfa3a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdf8f9e2700 1 -- 172.21.15.167:0/2502811804 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bfa702c0 con 0x5650bfa58c00 2024-09-13T23:43:53.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdf8f9e2700 1 -- 172.21.15.167:0/2502811804 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650bf99a300 con 0x5650bfa58c00 2024-09-13T23:43:53.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5650bf5fc340 con 0x5650bfa58c00 2024-09-13T23:43:53.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdf8f9e2700 1 -- 172.21.15.167:0/2502811804 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5650bfa70000 con 0x5650bfa58c00 2024-09-13T23:43:53.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.323+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650bf5fcb60 con 0x5650bfa58c00 2024-09-13T23:43:53.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.327+0000 7fdf8f9e2700 1 -- 172.21.15.167:0/2502811804 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5650bf6e5b80 con 0x5650bfa58c00 2024-09-13T23:43:53.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.327+0000 7fdf8f9e2700 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bfaec000 0x5650bfaf2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.327+0000 7fdf8f9e2700 1 -- 172.21.15.167:0/2502811804 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5650bf651500 con 0x5650bfa58c00 2024-09-13T23:43:53.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.327+0000 7fdfab219700 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bfaec000 0x5650bfaf2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.327+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5650bfa59000 0x5650bf8d3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.327+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa59400 con 0x5650bfa59000 2024-09-13T23:43:53.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.327+0000 7fdfaaa18700 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5650bfa59000 0x5650bf8d3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.327+0000 7fdfab219700 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bfaec000 0x5650bfaf2000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5650c045eed0 tx=0x5650bf94f500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.327+0000 7fdfaaa18700 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5650bfa59000 0x5650bf8d3700 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.331+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v35'37 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.331+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5650bfa58000 0x5650bfaf2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.331+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa58400 con 0x5650bfa58000 2024-09-13T23:43:53.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.331+0000 7fdfaa217700 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5650bfa58000 0x5650bfaf2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.331+0000 7fdfaa217700 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5650bfa58000 0x5650bfaf2580 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.335+0000 7fdfaa217700 1 -- 172.21.15.167:0/2502811804 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v35'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa58000 2024-09-13T23:43:53.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.335+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5650bfa52800 0x5650bfaf2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.335+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53000 con 0x5650bfa52800 2024-09-13T23:43:53.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.335+0000 7fdfab219700 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5650bfa52800 0x5650bfaf2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.335+0000 7fdfab219700 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5650bfa52800 0x5650bfaf2b00 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.335+0000 7fdfab219700 1 -- 172.21.15.167:0/2502811804 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v35'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52800 2024-09-13T23:43:53.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.335+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5650bf989800 0x5650bfaf3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:43:53.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.335+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaec400 con 0x5650bf989800 2024-09-13T23:43:53.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.335+0000 7fdfaaa18700 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5650bf989800 0x5650bfaf3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:43:53.346 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.339+0000 7fdfaaa18700 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5650bf989800 0x5650bfaf3080 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:43:53.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.339+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v35'192 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989800 2024-09-13T23:43:53.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.339+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaec800 con 0x5650bfa59000 2024-09-13T23:43:53.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.343+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v35'65 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.343+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaecc00 con 0x5650bfa52800 2024-09-13T23:43:53.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.343+0000 7fdfab219700 1 -- 172.21.15.167:0/2502811804 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v35'46 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52800 2024-09-13T23:43:53.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.343+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaed000 con 0x5650bfa59000 2024-09-13T23:43:53.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.347+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v35'27 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.356 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.347+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaed400 con 0x5650bf989800 2024-09-13T23:43:53.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.351+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v35'44 uv35 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989800 2024-09-13T23:43:53.359 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.351+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaed800 con 0x5650bfa59000 2024-09-13T23:43:53.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.351+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v35'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.351+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaedc00 con 0x5650bfa59000 2024-09-13T23:43:53.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.355+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v35'32 uv28 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.355+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa59400 con 0x5650bfa59000 2024-09-13T23:43:53.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.359+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v35'66 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.359+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53c00 con 0x5650bfa52800 2024-09-13T23:43:53.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.359+0000 7fdfab219700 1 -- 172.21.15.167:0/2502811804 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v35'28 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52800 2024-09-13T23:43:53.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.359+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53800 con 0x5650bfa59000 2024-09-13T23:43:53.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.363+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v35'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.363+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53400 con 0x5650bf989800 2024-09-13T23:43:53.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.367+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v35'49 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989800 2024-09-13T23:43:53.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.367+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa52c00 con 0x5650bf989800 2024-09-13T23:43:53.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.367+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v35'193 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989800 2024-09-13T23:43:53.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.367+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989c00 con 0x5650bfa59000 2024-09-13T23:43:53.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.371+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v35'62 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.371+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa59400 con 0x5650bfa59000 2024-09-13T23:43:53.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.375+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v35'67 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.375+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaec400 con 0x5650bfa52800 2024-09-13T23:43:53.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.379+0000 7fdfab219700 1 -- 172.21.15.167:0/2502811804 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v35'27 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52800 2024-09-13T23:43:53.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.379+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaec800 con 0x5650bfa59000 2024-09-13T23:43:53.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.379+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v35'34 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.379+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaed000 con 0x5650bf989800 2024-09-13T23:43:53.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.383+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v35'194 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989800 2024-09-13T23:43:53.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.383+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaed400 con 0x5650bfa59000 2024-09-13T23:43:53.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.387+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v35'63 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.387+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaed800 con 0x5650bfa59000 2024-09-13T23:43:53.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.391+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v35'35 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.391+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaedc00 con 0x5650bfa52800 2024-09-13T23:43:53.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.391+0000 7fdfab219700 1 -- 172.21.15.167:0/2502811804 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v35'27 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52800 2024-09-13T23:43:53.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.391+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaecc00 con 0x5650bfa59000 2024-09-13T23:43:53.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.395+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v35'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.395+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaec400 con 0x5650bfa52800 2024-09-13T23:43:53.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.399+0000 7fdfab219700 1 -- 172.21.15.167:0/2502811804 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v35'21 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52800 2024-09-13T23:43:53.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.399+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaedc00 con 0x5650bfa52800 2024-09-13T23:43:53.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.399+0000 7fdfab219700 1 -- 172.21.15.167:0/2502811804 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v35'47 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52800 2024-09-13T23:43:53.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.399+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaec800 con 0x5650bfa52800 2024-09-13T23:43:53.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.403+0000 7fdfab219700 1 -- 172.21.15.167:0/2502811804 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v35'27 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52800 2024-09-13T23:43:53.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.403+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaed000 con 0x5650bfa59000 2024-09-13T23:43:53.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.407+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v35'25 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.407+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfaed400 con 0x5650bf989800 2024-09-13T23:43:53.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.407+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v35'45 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989800 2024-09-13T23:43:53.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.407+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa59800 con 0x5650bfa59000 2024-09-13T23:43:53.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.411+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v35'64 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bfa59000 2024-09-13T23:43:53.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.411+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa59400 con 0x5650bf989800 2024-09-13T23:43:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.415+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v35'50 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989800 2024-09-13T23:43:53.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.415+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf989c00 con 0x5650bfa58000 2024-09-13T23:43:53.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.415+0000 7fdfaa217700 1 -- 172.21.15.167:0/2502811804 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v35'31 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa58000 2024-09-13T23:43:53.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.419+0000 7fdf8d1dd700 20 reqs_thread_entry: start 2024-09-13T23:43:53.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.419+0000 7fdfac25ff40 20 init_complete bucket index max shards: 11 2024-09-13T23:43:53.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.419+0000 7fdfac25ff40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:43:53.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.419+0000 7fdf8b1d9700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:23 2024 2024-09-13T23:43:53.429 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:43:53.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.419+0000 7fdf8a9b7700 20 reqs_thread_entry: start 2024-09-13T23:43:53.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.419+0000 7fdfac25ff40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:43:53.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.419+0000 7fdfac25ff40 20 rados->read ofs=0 len=0 2024-09-13T23:43:53.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.419+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53800 con 0x5650bf989000 2024-09-13T23:43:53.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.423+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.423+0000 7fdfac25ff40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:43:53.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.423+0000 7fdfac25ff40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:43:53.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.423+0000 7fdfac25ff40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:43:53.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.423+0000 7fdfac25ff40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:43:53.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.423+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa53400 con 0x5650bfa52400 2024-09-13T23:43:53.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.423+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv40 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.431 INFO:teuthology.orchestra.run.smithi167.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-09-13T23:43:53.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.423+0000 7fdf919e6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:43:53.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.423+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94904812691456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bfa52c00 con 0x5650bf989000 2024-09-13T23:43:53.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.427+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94904812691456] v35'195 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.427+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94904811027200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf9b2000 con 0x5650bf989400 2024-09-13T23:43:53.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.431+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94904811027200] v35'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.431+0000 7fdfac25ff40 20 remove_watcher() i=0 2024-09-13T23:43:53.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.431+0000 7fdfac25ff40 2 removed watcher, disabling cache 2024-09-13T23:43:53.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.431+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94904811028352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf9b2400 con 0x5650bf989000 2024-09-13T23:43:53.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.431+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94904811028352] v35'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bf989000 2024-09-13T23:43:53.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.431+0000 7fdfac25ff40 20 remove_watcher() i=1 2024-09-13T23:43:53.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.431+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94904811029504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf9b2800 con 0x5650bf989400 2024-09-13T23:43:53.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.435+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94904811029504] v35'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.435+0000 7fdfac25ff40 20 remove_watcher() i=2 2024-09-13T23:43:53.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.435+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94904811030656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf9b2c00 con 0x5650bfa52400 2024-09-13T23:43:53.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.439+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94904811030656] v35'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.439+0000 7fdfac25ff40 20 remove_watcher() i=3 2024-09-13T23:43:53.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.439+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94904811031808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf9b3000 con 0x5650bf989400 2024-09-13T23:43:53.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.439+0000 7fdfaaa18700 1 -- 172.21.15.167:0/3465567726 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94904811031808] v35'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05200 con 0x5650bf989400 2024-09-13T23:43:53.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.439+0000 7fdfac25ff40 20 remove_watcher() i=4 2024-09-13T23:43:53.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.439+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94904811032960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf9b3400 con 0x5650bfa52000 2024-09-13T23:43:53.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.443+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94904811032960] v35'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.443+0000 7fdfac25ff40 20 remove_watcher() i=5 2024-09-13T23:43:53.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.443+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94904811034112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf9b3800 con 0x5650bfa52000 2024-09-13T23:43:53.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.443+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94904811034112] v35'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa05440 con 0x5650bfa52000 2024-09-13T23:43:53.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.443+0000 7fdfac25ff40 20 remove_watcher() i=6 2024-09-13T23:43:53.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.443+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94904811035264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5650bf9b3c00 con 0x5650bfa52400 2024-09-13T23:43:53.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfab219700 1 -- 172.21.15.167:0/3465567726 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94904811035264] v35'188 uv186 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5650bfa04b40 con 0x5650bfa52400 2024-09-13T23:43:53.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 20 remove_watcher() i=7 2024-09-13T23:43:53.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5650bf989400 msgr2=0x5650bf8d2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5650bf989400 0x5650bf8d2100 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5650bf989000 msgr2=0x5650bf8d1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5650bf989000 0x5650bf8d1b80 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5650bfa52400 msgr2=0x5650bf8d2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5650bfa52400 0x5650bf8d2c00 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5650bfa52000 msgr2=0x5650bf8d2680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5650bfa52000 0x5650bf8d2680 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bf988c00 msgr2=0x5650bf8d1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bf988c00 0x5650bf8d1600 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x5650bf98bec0 tx=0x5650bfa3a0c0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 msgr2=0x5650bf8d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfaa217700 1 -- 172.21.15.167:0/3465567726 reap_dead start 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 0x5650bf8d0b00 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x5650c045e2a0 tx=0x5650bf6efce0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 shutdown_connections 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/3465567726 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb800 0x5650bf8d0b00 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 >> 172.21.15.167:0/3465567726 conn(0x5650bf5eb400 msgr2=0x5650bf6eb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 shutdown_connections 2024-09-13T23:43:53.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/3465567726 wait complete. 2024-09-13T23:43:53.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5650bfa59000 msgr2=0x5650bf8d3700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5650bfa59000 0x5650bf8d3700 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5650bf989800 msgr2=0x5650bfaf3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5650bf989800 0x5650bfaf3080 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5650bfa58000 msgr2=0x5650bfaf2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5650bfa58000 0x5650bfaf2580 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.447+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5650bfa52800 msgr2=0x5650bfaf2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5650bfa52800 0x5650bfaf2b00 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bfaec000 msgr2=0x5650bfaf2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bfaec000 0x5650bfaf2000 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x5650c045eed0 tx=0x5650bf94f500 comp rx=0 tx=0).stop 2024-09-13T23:43:53.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 msgr2=0x5650bf8d3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfaaa18700 1 -- 172.21.15.167:0/2502811804 reap_dead start 2024-09-13T23:43:53.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 0x5650bf8d3180 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x5650bfa69350 tx=0x5650bfa3a180 comp rx=0 tx=0).stop 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 shutdown_connections 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2502811804 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bfa58c00 0x5650bf8d3180 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 >> 172.21.15.167:0/2502811804 conn(0x5650bfa59c00 msgr2=0x5650bf6ec000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 shutdown_connections 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2502811804 wait complete. 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2600688409 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bf988000 msgr2=0x5650bf8d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2600688409 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bf988000 0x5650bf8d1080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x5650c045f080 tx=0x5650bf94ec60 comp rx=0 tx=0).stop 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2600688409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 msgr2=0x5650bf8d0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2600688409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x5650c044f6b0 tx=0x5650bf6ef920 comp rx=0 tx=0).stop 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2600688409 shutdown_connections 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2600688409 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5650bf988000 0x5650bf8d1080 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 --2- 172.21.15.167:0/2600688409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5650bf5eb000 0x5650bf8d0580 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2600688409 >> 172.21.15.167:0/2600688409 conn(0x5650bf5eac00 msgr2=0x5650bf6eb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:43:53.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2600688409 shutdown_connections 2024-09-13T23:43:53.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:43:53.451+0000 7fdfac25ff40 1 -- 172.21.15.167:0/2600688409 wait complete. 2024-09-13T23:43:53.465 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-09-13T23:43:53.471 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:43:53.517 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-09-13T23:44:28.518 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--categories', 'delete_obj'] 2024-09-13T23:44:28.519 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--categories', 'delete_obj'] 2024-09-13T23:44:28.519 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --categories delete_obj 2024-09-13T23:44:28.562 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:28.562 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:28.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.557+0000 7f24916e3f40 1 Processor -- start 2024-09-13T23:44:28.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.557+0000 7f24916e3f40 1 -- start start 2024-09-13T23:44:28.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.557+0000 7f24916e3f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55840dbdf400 0x55840dec4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.557+0000 7f24916e3f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55840dca3500 con 0x55840dbdf000 2024-09-13T23:44:28.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.557+0000 7f24916e3f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55840dc6ca00 con 0x55840dbdf400 2024-09-13T23:44:28.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.557+0000 7f248f69b700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55840dbdf400 0x55840dec4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.557+0000 7f248fe9c700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55840dbdf000 0x55840dcde800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:36348/0 (socket says 172.21.15.167:36348) 2024-09-13T23:44:28.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.557+0000 7f248fe9c700 1 -- 172.21.15.167:0/3363555814 learned_addr learned my addr 172.21.15.167:0/3363555814 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:44:28.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f248f69b700 1 -- 172.21.15.167:0/3363555814 >> v1:172.21.15.12:6789/0 conn(0x55840dbdf000 legacy=0x55840dcde800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f248f69b700 1 -- 172.21.15.167:0/3363555814 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840dbf1d40 con 0x55840dbdf400 2024-09-13T23:44:28.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f248f69b700 1 --2- 172.21.15.167:0/3363555814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf400 0x55840dec4580 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55840debf920 tx=0x55840dce3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c87057b96d6cb3b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f248ee9a700 1 -- 172.21.15.167:0/3363555814 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840dc834a0 con 0x55840dbdf400 2024-09-13T23:44:28.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f248ee9a700 1 -- 172.21.15.167:0/3363555814 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55840dca3800 con 0x55840dbdf400 2024-09-13T23:44:28.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f248ee9a700 1 -- 172.21.15.167:0/3363555814 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840dc83600 con 0x55840dbdf400 2024-09-13T23:44:28.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f24916e3f40 1 -- 172.21.15.167:0/3363555814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf400 msgr2=0x55840dec4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3363555814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf400 0x55840dec4580 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x55840debf920 tx=0x55840dce3920 comp rx=0 tx=0).stop 2024-09-13T23:44:28.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f24916e3f40 1 -- 172.21.15.167:0/3363555814 shutdown_connections 2024-09-13T23:44:28.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3363555814 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf400 0x55840dec4580 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f24916e3f40 1 -- 172.21.15.167:0/3363555814 >> 172.21.15.167:0/3363555814 conn(0x55840dbdec00 msgr2=0x55840dcde000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f24916e3f40 1 -- 172.21.15.167:0/3363555814 shutdown_connections 2024-09-13T23:44:28.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f24916e3f40 1 -- 172.21.15.167:0/3363555814 wait complete. 2024-09-13T23:44:28.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f24916e3f40 1 Processor -- start 2024-09-13T23:44:28.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f24916e3f40 1 -- start start 2024-09-13T23:44:28.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f24916e3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 0x55840dec4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f24916e3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840dc6c500 con 0x55840dbdf000 2024-09-13T23:44:28.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f248fe9c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 0x55840dec4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f248fe9c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 0x55840dec4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42380/0 (socket says 172.21.15.167:42380) 2024-09-13T23:44:28.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.561+0000 7f248fe9c700 1 -- 172.21.15.167:0/2629309023 learned_addr learned my addr 172.21.15.167:0/2629309023 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f248fe9c700 1 -- 172.21.15.167:0/2629309023 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840dbf1ba0 con 0x55840dbdf000 2024-09-13T23:44:28.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f248fe9c700 1 --2- 172.21.15.167:0/2629309023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 0x55840dec4580 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x55840ea3f8c0 tx=0x55840dce3b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96535c761c6b81d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f248e699700 1 -- 172.21.15.167:0/2629309023 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840dc834a0 con 0x55840dbdf000 2024-09-13T23:44:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f248e699700 1 -- 172.21.15.167:0/2629309023 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55840dca3b00 con 0x55840dbdf000 2024-09-13T23:44:28.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f248e699700 1 -- 172.21.15.167:0/2629309023 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840dc83600 con 0x55840dbdf000 2024-09-13T23:44:28.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f24916e3f40 1 -- 172.21.15.167:0/2629309023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 msgr2=0x55840dec4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f24916e3f40 1 --2- 172.21.15.167:0/2629309023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 0x55840dec4580 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x55840ea3f8c0 tx=0x55840dce3b60 comp rx=0 tx=0).stop 2024-09-13T23:44:28.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f24916e3f40 1 -- 172.21.15.167:0/2629309023 shutdown_connections 2024-09-13T23:44:28.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f24916e3f40 1 --2- 172.21.15.167:0/2629309023 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 0x55840dec4580 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f24916e3f40 1 -- 172.21.15.167:0/2629309023 >> 172.21.15.167:0/2629309023 conn(0x55840dbdec00 msgr2=0x55840dcdf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f24916e3f40 1 -- 172.21.15.167:0/2629309023 shutdown_connections 2024-09-13T23:44:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f24916e3f40 1 -- 172.21.15.167:0/2629309023 wait complete. 2024-09-13T23:44:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f24916e3f40 1 Processor -- start 2024-09-13T23:44:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f24916e3f40 1 -- start start 2024-09-13T23:44:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f24916e3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 0x55840dec4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f24916e3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840dc6c3c0 con 0x55840dbdf000 2024-09-13T23:44:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f248fe9c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 0x55840dec4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f248fe9c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 0x55840dec4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42384/0 (socket says 172.21.15.167:42384) 2024-09-13T23:44:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.565+0000 7f248fe9c700 1 -- 172.21.15.167:0/4119238305 learned_addr learned my addr 172.21.15.167:0/4119238305 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f248fe9c700 1 -- 172.21.15.167:0/4119238305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840dbf1a00 con 0x55840dbdf000 2024-09-13T23:44:28.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f248fe9c700 1 --2- 172.21.15.167:0/4119238305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 0x55840dec4580 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x55840ea3faa0 tx=0x55840dce3a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f248ce96700 1 -- 172.21.15.167:0/4119238305 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840dc838c0 con 0x55840dbdf000 2024-09-13T23:44:28.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f248ce96700 1 -- 172.21.15.167:0/4119238305 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55840dca3c80 con 0x55840dbdf000 2024-09-13T23:44:28.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f24916e3f40 1 -- 172.21.15.167:0/4119238305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55840dbf1860 con 0x55840dbdf000 2024-09-13T23:44:28.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f248ce96700 1 -- 172.21.15.167:0/4119238305 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840dc834a0 con 0x55840dbdf000 2024-09-13T23:44:28.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f24916e3f40 1 -- 172.21.15.167:0/4119238305 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55840dbf16c0 con 0x55840dbdf000 2024-09-13T23:44:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f24916e3f40 1 Processor -- start 2024-09-13T23:44:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f24916e3f40 1 -- start start 2024-09-13T23:44:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f24916e3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 0x55840dec4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f24916e3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840dc6c000 con 0x55840dbdf800 2024-09-13T23:44:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f249069d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 0x55840dec4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f249069d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 0x55840dec4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42388/0 (socket says 172.21.15.167:42388) 2024-09-13T23:44:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f249069d700 1 -- 172.21.15.167:0/2344180979 learned_addr learned my addr 172.21.15.167:0/2344180979 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f248ce96700 1 -- 172.21.15.167:0/4119238305 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55840dcd8b00 con 0x55840dbdf000 2024-09-13T23:44:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f249069d700 1 -- 172.21.15.167:0/2344180979 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840dbf0000 con 0x55840dbdf800 2024-09-13T23:44:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f249069d700 1 --2- 172.21.15.167:0/2344180979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 0x55840dec4b00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55840ea57110 tx=0x55840df4a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f7937f2c0d0201 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f248ce96700 1 --2- 172.21.15.167:0/4119238305 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840df3a000 0x55840dec5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f248ce96700 1 -- 172.21.15.167:0/4119238305 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55840dc45180 con 0x55840dbdf000 2024-09-13T23:44:28.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f248a691700 1 -- 172.21.15.167:0/2344180979 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840dc838c0 con 0x55840dbdf800 2024-09-13T23:44:28.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f248a691700 1 -- 172.21.15.167:0/2344180979 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55840df96000 con 0x55840dbdf800 2024-09-13T23:44:28.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f248fe9c700 1 --2- 172.21.15.167:0/4119238305 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840df3a000 0x55840dec5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.569+0000 7f248a691700 1 -- 172.21.15.167:0/2344180979 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840dc834a0 con 0x55840dbdf800 2024-09-13T23:44:28.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f24916e3f40 1 -- 172.21.15.167:0/2344180979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 msgr2=0x55840dec4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f248fe9c700 1 --2- 172.21.15.167:0/4119238305 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840df3a000 0x55840dec5080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55840ea576e0 tx=0x55840df4aea0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f24916e3f40 1 --2- 172.21.15.167:0/2344180979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 0x55840dec4b00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x55840ea57110 tx=0x55840df4a660 comp rx=0 tx=0).stop 2024-09-13T23:44:28.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f24916e3f40 1 -- 172.21.15.167:0/2344180979 shutdown_connections 2024-09-13T23:44:28.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f24916e3f40 1 --2- 172.21.15.167:0/2344180979 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 0x55840dec4b00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f24916e3f40 1 -- 172.21.15.167:0/2344180979 >> 172.21.15.167:0/2344180979 conn(0x55840dbdf400 msgr2=0x55840dcdf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f24916e3f40 1 -- 172.21.15.167:0/2344180979 shutdown_connections 2024-09-13T23:44:28.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f24916e3f40 1 -- 172.21.15.167:0/2344180979 wait complete. 2024-09-13T23:44:28.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f24916e3f40 1 Processor -- start 2024-09-13T23:44:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f24916e3f40 1 -- start start 2024-09-13T23:44:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f24916e3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 0x55840dec4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f24916e3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840dc04640 con 0x55840dbdf800 2024-09-13T23:44:28.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f249069d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 0x55840dec4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f249069d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 0x55840dec4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42392/0 (socket says 172.21.15.167:42392) 2024-09-13T23:44:28.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 learned_addr learned my addr 172.21.15.167:0/3969219575 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840dbf0ea0 con 0x55840dbdf800 2024-09-13T23:44:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.573+0000 7f249069d700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 0x55840dec4b00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55840ea56e10 tx=0x55840dce3da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f2488e8e700 1 -- 172.21.15.167:0/3969219575 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840dc83b80 con 0x55840dbdf800 2024-09-13T23:44:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55840dbf0820 con 0x55840dbdf800 2024-09-13T23:44:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f2488e8e700 1 -- 172.21.15.167:0/3969219575 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55840df96180 con 0x55840dbdf800 2024-09-13T23:44:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f2488e8e700 1 -- 172.21.15.167:0/3969219575 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840dc838c0 con 0x55840dbdf800 2024-09-13T23:44:28.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55840dbf09c0 con 0x55840dbdf800 2024-09-13T23:44:28.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f2488e8e700 1 -- 172.21.15.167:0/3969219575 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55840dcd9080 con 0x55840dbdf800 2024-09-13T23:44:28.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f2488e8e700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840df3ac00 0x55840dec5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f2488e8e700 1 -- 172.21.15.167:0/3969219575 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55840dc45340 con 0x55840dbdf800 2024-09-13T23:44:28.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f248f69b700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840df3ac00 0x55840dec5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f24916e3f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55840df3b000 0x55840dec5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b400 con 0x55840df3b000 2024-09-13T23:44:28.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f249069d700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55840df3b000 0x55840dec5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f248f69b700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840df3ac00 0x55840dec5600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55840debf920 tx=0x55840dce3800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.577+0000 7f249069d700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55840df3b000 0x55840dec5b80 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 20 realm 2024-09-13T23:44:28.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b000 2024-09-13T23:44:28.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:28.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3bc00 con 0x55840df3b000 2024-09-13T23:44:28.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55840df3b400 0x55840dec6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b400 2024-09-13T23:44:28.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f248fe9c700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55840df3b400 0x55840dec6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f248fe9c700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55840df3b400 0x55840dec6100 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:28.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3bc00 con 0x55840df3b400 2024-09-13T23:44:28.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:28.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 20 searching for the correct realm 2024-09-13T23:44:28.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55840defe000 0x55840dec6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840defe400 con 0x55840defe000 2024-09-13T23:44:28.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.581+0000 7f248f69b700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55840defe000 0x55840dec6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f248f69b700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55840defe000 0x55840dec6680 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840defe400 con 0x55840df3b400 2024-09-13T23:44:28.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f248fe9c700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55840defe400 0x55840dec6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe400 2024-09-13T23:44:28.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f249069d700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55840defe400 0x55840dec6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f249069d700 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55840defe400 0x55840dec6c00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b000 2024-09-13T23:44:28.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b000 2024-09-13T23:44:28.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.585+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe000 2024-09-13T23:44:28.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.589+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.589+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b000 2024-09-13T23:44:28.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.589+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.589+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b400 2024-09-13T23:44:28.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.589+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.589+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b400 2024-09-13T23:44:28.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.589+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.589+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b000 2024-09-13T23:44:28.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.589+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.589+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe400 2024-09-13T23:44:28.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.589+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.589+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b000 2024-09-13T23:44:28.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe400 2024-09-13T23:44:28.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b400 2024-09-13T23:44:28.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe000 2024-09-13T23:44:28.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b000 2024-09-13T23:44:28.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b400 2024-09-13T23:44:28.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe000 2024-09-13T23:44:28.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.593+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b000 2024-09-13T23:44:28.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe400 2024-09-13T23:44:28.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b000 2024-09-13T23:44:28.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe000 2024-09-13T23:44:28.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe400 2024-09-13T23:44:28.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b000 2024-09-13T23:44:28.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe400 2024-09-13T23:44:28.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.597+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe000 2024-09-13T23:44:28.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.601+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.601+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe400 2024-09-13T23:44:28.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.601+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.601+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b400 2024-09-13T23:44:28.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.601+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.601+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b400 2024-09-13T23:44:28.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.601+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.601+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe000 2024-09-13T23:44:28.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.601+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.601+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b400 2024-09-13T23:44:28.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.601+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.601+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b400 2024-09-13T23:44:28.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:28.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:28.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:28.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:28.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840defe800 con 0x55840df3b000 2024-09-13T23:44:28.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840defec00 con 0x55840df3b400 2024-09-13T23:44:28.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:28.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff000 con 0x55840df3b400 2024-09-13T23:44:28.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:28.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 zone default found 2024-09-13T23:44:28.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 4 Realm: () 2024-09-13T23:44:28.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:28.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:28.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:28.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 zonegroup default 2024-09-13T23:44:28.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff400 con 0x55840defe400 2024-09-13T23:44:28.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff800 con 0x55840defe000 2024-09-13T23:44:28.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.605+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.609+0000 7f24916e3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.609+0000 7f24916e3f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.609+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deffc00 con 0x55840defe000 2024-09-13T23:44:28.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.609+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.609+0000 7f24916e3f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.609+0000 7f24916e3f40 20 started sync module instance, tier type = 2024-09-13T23:44:28.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.609+0000 7f24916e3f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:28.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.609+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff800 con 0x55840df3b400 2024-09-13T23:44:28.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.609+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.609+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94025657596672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deffc00 con 0x55840df3b400 2024-09-13T23:44:28.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.609+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840defec00 con 0x55840df3b000 2024-09-13T23:44:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.613+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.613+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94025657597824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff000 con 0x55840df3b000 2024-09-13T23:44:28.618 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.613+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff800 con 0x55840df3b400 2024-09-13T23:44:28.619 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.613+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94025657596672] v35'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.617+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94025657597824] v35'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.617+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.617+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94025657598976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3bc00 con 0x55840df3b400 2024-09-13T23:44:28.621 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.617+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe400 2024-09-13T23:44:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.617+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.617+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94025657600128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840defe800 con 0x55840defe400 2024-09-13T23:44:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.617+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff400 con 0x55840df3b400 2024-09-13T23:44:28.624 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.621+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94025657598976] v35'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.621+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94025657600128] v35'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.621+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.621+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94025657601280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840defec00 con 0x55840df3b400 2024-09-13T23:44:28.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.621+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff000 con 0x55840defe000 2024-09-13T23:44:28.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.625+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.625+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94025657602432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e052000 con 0x55840defe000 2024-09-13T23:44:28.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.625+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e052400 con 0x55840defe000 2024-09-13T23:44:28.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.625+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94025657601280] v35'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.625+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94025657602432] v35'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.625+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.625+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94025657603584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e052800 con 0x55840defe000 2024-09-13T23:44:28.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.625+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e052c00 con 0x55840defe400 2024-09-13T23:44:28.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94025657603584] v35'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'189 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94025657604736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e053000 con 0x55840defe400 2024-09-13T23:44:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f24916e3f40 20 add_watcher() i=0 2024-09-13T23:44:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f24916e3f40 20 add_watcher() i=1 2024-09-13T23:44:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f24916e3f40 20 add_watcher() i=2 2024-09-13T23:44:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f24916e3f40 20 add_watcher() i=3 2024-09-13T23:44:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f24916e3f40 20 add_watcher() i=4 2024-09-13T23:44:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f24916e3f40 20 add_watcher() i=5 2024-09-13T23:44:28.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f24916e3f40 20 add_watcher() i=6 2024-09-13T23:44:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94025657604736] v35'190 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f24916e3f40 20 add_watcher() i=7 2024-09-13T23:44:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f24916e3f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e053400 con 0x55840df3b000 2024-09-13T23:44:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.629+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94025659285504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e053800 con 0x55840df3b000 2024-09-13T23:44:28.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94025659285504] v35'196 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f24916e3f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:28.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f24916e3f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:28.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f24916e3f40 1 Processor -- start 2024-09-13T23:44:28.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f2476e6a700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:28.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f24916e3f40 1 -- start start 2024-09-13T23:44:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f24916e3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 0x55840dec7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f24916e3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840dc04780 con 0x55840e052c00 2024-09-13T23:44:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f248f69b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 0x55840dec7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f248f69b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 0x55840dec7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42396/0 (socket says 172.21.15.167:42396) 2024-09-13T23:44:28.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f248f69b700 1 -- 172.21.15.167:0/1735782181 learned_addr learned my addr 172.21.15.167:0/1735782181 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f248f69b700 1 -- 172.21.15.167:0/1735782181 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840dbf0b60 con 0x55840e052c00 2024-09-13T23:44:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f248f69b700 1 --2- 172.21.15.167:0/1735782181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 0x55840dec7180 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x55840e0600f0 tx=0x55840dce3860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e57829f0d25ce92 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.633+0000 7f2476669700 1 -- 172.21.15.167:0/1735782181 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840dc83760 con 0x55840e052c00 2024-09-13T23:44:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f2476669700 1 -- 172.21.15.167:0/1735782181 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55840df96600 con 0x55840e052c00 2024-09-13T23:44:28.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f2476669700 1 -- 172.21.15.167:0/1735782181 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840e064000 con 0x55840e052c00 2024-09-13T23:44:28.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 -- 172.21.15.167:0/1735782181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 msgr2=0x55840dec7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 --2- 172.21.15.167:0/1735782181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 0x55840dec7180 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x55840e0600f0 tx=0x55840dce3860 comp rx=0 tx=0).stop 2024-09-13T23:44:28.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 -- 172.21.15.167:0/1735782181 shutdown_connections 2024-09-13T23:44:28.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 --2- 172.21.15.167:0/1735782181 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 0x55840dec7180 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 -- 172.21.15.167:0/1735782181 >> 172.21.15.167:0/1735782181 conn(0x55840e053c00 msgr2=0x55840dce0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 -- 172.21.15.167:0/1735782181 shutdown_connections 2024-09-13T23:44:28.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 -- 172.21.15.167:0/1735782181 wait complete. 2024-09-13T23:44:28.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 Processor -- start 2024-09-13T23:44:28.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 -- start start 2024-09-13T23:44:28.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 0x55840dec7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840dc6c000 con 0x55840e052c00 2024-09-13T23:44:28.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f248f69b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 0x55840dec7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f248f69b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 0x55840dec7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42410/0 (socket says 172.21.15.167:42410) 2024-09-13T23:44:28.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f248f69b700 1 -- 172.21.15.167:0/3486851676 learned_addr learned my addr 172.21.15.167:0/3486851676 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f248f69b700 1 -- 172.21.15.167:0/3486851676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840dbf0d00 con 0x55840e052c00 2024-09-13T23:44:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f248f69b700 1 --2- 172.21.15.167:0/3486851676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 0x55840dec7180 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55840e060390 tx=0x55840e02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f2474e66700 1 -- 172.21.15.167:0/3486851676 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840e064160 con 0x55840e052c00 2024-09-13T23:44:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f2474e66700 1 -- 172.21.15.167:0/3486851676 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55840df96300 con 0x55840e052c00 2024-09-13T23:44:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 -- 172.21.15.167:0/3486851676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55840dbf0340 con 0x55840e052c00 2024-09-13T23:44:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f2474e66700 1 -- 172.21.15.167:0/3486851676 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55840e0642c0 con 0x55840e052c00 2024-09-13T23:44:28.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.637+0000 7f24916e3f40 1 -- 172.21.15.167:0/3486851676 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55840dbf1d40 con 0x55840e052c00 2024-09-13T23:44:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.641+0000 7f2474e66700 1 -- 172.21.15.167:0/3486851676 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55840dcd9b80 con 0x55840e052c00 2024-09-13T23:44:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.641+0000 7f2474e66700 1 --2- 172.21.15.167:0/3486851676 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840e0e0000 0x55840e0e6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.641+0000 7f2474e66700 1 -- 172.21.15.167:0/3486851676 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55840dc45500 con 0x55840e052c00 2024-09-13T23:44:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.641+0000 7f24916e3f40 5 note: GC not initialized 2024-09-13T23:44:28.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.641+0000 7f249069d700 1 --2- 172.21.15.167:0/3486851676 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840e0e0000 0x55840e0e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.641+0000 7f249069d700 1 --2- 172.21.15.167:0/3486851676 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840e0e0000 0x55840e0e6000 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55840dfdfd10 tx=0x55840df4b680 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f2472661700 20 reqs_thread_entry: start 2024-09-13T23:44:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f24916e3f40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f24916e3f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f247065d700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:59 2024 2024-09-13T23:44:28.649 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f246fe3b700 20 reqs_thread_entry: start 2024-09-13T23:44:28.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e052000 con 0x55840df3b400 2024-09-13T23:44:28.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e052400 con 0x55840df3b400 2024-09-13T23:44:28.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840defe800 con 0x55840df3b400 2024-09-13T23:44:28.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff000 con 0x55840defe000 2024-09-13T23:44:28.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.645+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840df3b400 2024-09-13T23:44:28.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e0400 con 0x55840defe000 2024-09-13T23:44:28.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e0800 con 0x55840df3b000 2024-09-13T23:44:28.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e0c00 con 0x55840df3b000 2024-09-13T23:44:28.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e1000 con 0x55840defe000 2024-09-13T23:44:28.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e1400 con 0x55840defe000 2024-09-13T23:44:28.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e1800 con 0x55840defe000 2024-09-13T23:44:28.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(72 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.649+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e1c00 con 0x55840df3b000 2024-09-13T23:44:28.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.653+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.653+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e052000 con 0x55840defe400 2024-09-13T23:44:28.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.653+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.653+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deffc00 con 0x55840defe000 2024-09-13T23:44:28.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.653+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.653+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff800 con 0x55840defe400 2024-09-13T23:44:28.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.653+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.653+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff400 con 0x55840df3b000 2024-09-13T23:44:28.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.653+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.653+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840defec00 con 0x55840defe000 2024-09-13T23:44:28.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.653+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.653+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3bc00 con 0x55840df3b000 2024-09-13T23:44:28.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(79 usage.17 [call out=163b] v0'0 uv5 ondisk = 0) v8 ==== 152+0+163 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e0800 con 0x55840df3b400 2024-09-13T23:44:28.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e0c00 con 0x55840df3b400 2024-09-13T23:44:28.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e1c00 con 0x55840defe000 2024-09-13T23:44:28.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e0400 con 0x55840df3b400 2024-09-13T23:44:28.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e1000 con 0x55840defe000 2024-09-13T23:44:28.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e1400 con 0x55840df3b000 2024-09-13T23:44:28.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.657+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff800 con 0x55840df3b400 2024-09-13T23:44:28.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.661+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.661+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff000 con 0x55840df3b400 2024-09-13T23:44:28.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.661+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.661+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e053800 con 0x55840df3b400 2024-09-13T23:44:28.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.661+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.661+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e052000 con 0x55840df3b000 2024-09-13T23:44:28.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.661+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.661+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e0800 con 0x55840defe000 2024-09-13T23:44:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.661+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.661+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e0c00 con 0x55840defe000 2024-09-13T23:44:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.661+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.661+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e0400 con 0x55840df3b400 2024-09-13T23:44:28.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.665+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.665+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e1800 con 0x55840df3b000 2024-09-13T23:44:28.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.665+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.665+0000 7f2476e6a700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:28.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.665+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94025659285504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e1c00 con 0x55840df3b000 2024-09-13T23:44:28.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.669+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94025659285504] v35'197 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.669+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94025657596672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e0e1000 con 0x55840df3b400 2024-09-13T23:44:28.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.673+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 31 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94025657596672] v35'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.673+0000 7f24916e3f40 20 remove_watcher() i=0 2024-09-13T23:44:28.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.673+0000 7f24916e3f40 2 removed watcher, disabling cache 2024-09-13T23:44:28.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.673+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94025657597824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840defe800 con 0x55840df3b000 2024-09-13T23:44:28.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.673+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94025657597824] v35'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840df3b000 2024-09-13T23:44:28.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.673+0000 7f24916e3f40 20 remove_watcher() i=1 2024-09-13T23:44:28.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.673+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94025657598976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff800 con 0x55840df3b400 2024-09-13T23:44:28.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.677+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 32 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94025657598976] v35'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.677+0000 7f24916e3f40 20 remove_watcher() i=2 2024-09-13T23:44:28.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.677+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94025657600128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deffc00 con 0x55840defe400 2024-09-13T23:44:28.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.681+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94025657600128] v35'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.681+0000 7f24916e3f40 20 remove_watcher() i=3 2024-09-13T23:44:28.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.681+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94025657601280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840defec00 con 0x55840df3b400 2024-09-13T23:44:28.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.681+0000 7f248fe9c700 1 -- 172.21.15.167:0/3969219575 <== osd.0 v2:172.21.15.12:6800/2649399468 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94025657601280] v35'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043200 con 0x55840df3b400 2024-09-13T23:44:28.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.681+0000 7f24916e3f40 20 remove_watcher() i=4 2024-09-13T23:44:28.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.681+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94025657602432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840df3b800 con 0x55840defe000 2024-09-13T23:44:28.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.685+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94025657602432] v35'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.685+0000 7f24916e3f40 20 remove_watcher() i=5 2024-09-13T23:44:28.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.685+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94025657603584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840e052400 con 0x55840defe000 2024-09-13T23:44:28.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 <== osd.3 v2:172.21.15.167:6816/3688363524 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94025657603584] v35'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e043440 con 0x55840defe000 2024-09-13T23:44:28.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f24916e3f40 20 remove_watcher() i=6 2024-09-13T23:44:28.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94025657604736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55840deff400 con 0x55840defe400 2024-09-13T23:44:28.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f249069d700 1 -- 172.21.15.167:0/3969219575 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94025657604736] v35'191 uv189 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55840e042b40 con 0x55840defe400 2024-09-13T23:44:28.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f24916e3f40 20 remove_watcher() i=7 2024-09-13T23:44:28.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55840df3b400 msgr2=0x55840dec6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55840df3b400 0x55840dec6100 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55840df3b000 msgr2=0x55840dec5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55840df3b000 0x55840dec5b80 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55840defe400 msgr2=0x55840dec6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55840defe400 0x55840dec6c00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55840defe000 msgr2=0x55840dec6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.689+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55840defe000 0x55840dec6680 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840df3ac00 msgr2=0x55840dec5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840df3ac00 0x55840dec5600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x55840debf920 tx=0x55840dce3800 comp rx=0 tx=0).stop 2024-09-13T23:44:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 msgr2=0x55840dec4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 0x55840dec4b00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55840ea56e10 tx=0x55840dce3da0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f248f69b700 1 -- 172.21.15.167:0/3969219575 reap_dead start 2024-09-13T23:44:28.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 shutdown_connections 2024-09-13T23:44:28.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3969219575 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf800 0x55840dec4b00 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 >> 172.21.15.167:0/3969219575 conn(0x55840dbdf400 msgr2=0x55840dcdf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 shutdown_connections 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/3969219575 wait complete. 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/3486851676 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840e0e0000 msgr2=0x55840e0e6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3486851676 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840e0e0000 0x55840e0e6000 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55840dfdfd10 tx=0x55840df4b680 comp rx=0 tx=0).stop 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/3486851676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 msgr2=0x55840dec7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3486851676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 0x55840dec7180 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x55840e060390 tx=0x55840e02a420 comp rx=0 tx=0).stop 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/3486851676 shutdown_connections 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3486851676 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840e0e0000 0x55840e0e6000 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 --2- 172.21.15.167:0/3486851676 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840e052c00 0x55840dec7180 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/3486851676 >> 172.21.15.167:0/3486851676 conn(0x55840e053c00 msgr2=0x55840dce0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/3486851676 shutdown_connections 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/3486851676 wait complete. 2024-09-13T23:44:28.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/4119238305 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840df3a000 msgr2=0x55840dec5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 --2- 172.21.15.167:0/4119238305 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840df3a000 0x55840dec5080 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55840ea576e0 tx=0x55840df4aea0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/4119238305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 msgr2=0x55840dec4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 --2- 172.21.15.167:0/4119238305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 0x55840dec4580 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x55840ea3faa0 tx=0x55840dce3a40 comp rx=0 tx=0).stop 2024-09-13T23:44:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/4119238305 shutdown_connections 2024-09-13T23:44:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 --2- 172.21.15.167:0/4119238305 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55840df3a000 0x55840dec5080 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 --2- 172.21.15.167:0/4119238305 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55840dbdf000 0x55840dec4580 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/4119238305 >> 172.21.15.167:0/4119238305 conn(0x55840dbdec00 msgr2=0x55840dcdf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/4119238305 shutdown_connections 2024-09-13T23:44:28.701 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.693+0000 7f24916e3f40 1 -- 172.21.15.167:0/4119238305 wait complete. 2024-09-13T23:44:28.701 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:44:28.702 INFO:teuthology.orchestra.run.smithi167.stdout: "entries": [ 2024-09-13T23:44:28.702 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.702 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "anonymous", 2024-09-13T23:44:28.702 INFO:teuthology.orchestra.run.smithi167.stdout: "buckets": [ 2024-09-13T23:44:28.702 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.702 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "", 2024-09-13T23:44:28.702 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:28.702 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:28.702 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "anonymous", 2024-09-13T23:44:28.702 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:28.702 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.702 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: "buckets": [ 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "", 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo", 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:28.703 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_obj", 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 4, 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.704 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo2", 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo3", 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.705 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo4", 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: "summary": [ 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "anonymous", 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [], 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: "total": { 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.706 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 0, 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 0 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_obj", 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 4, 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:28.707 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.708 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:28.708 INFO:teuthology.orchestra.run.smithi167.stdout: "total": { 2024-09-13T23:44:28.708 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.708 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.708 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 4, 2024-09-13T23:44:28.708 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:28.708 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.708 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.708 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.708 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:44:28.709 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'anonymous', 'categories': []}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}]}, {'bucket': 'myfoo2', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}]} 2024-09-13T23:44:28.709 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-09-13T23:44:28.709 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-09-13T23:44:28.710 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-09-13T23:44:28.792 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:28.792 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:28.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb05abbdf40 1 Processor -- start 2024-09-13T23:44:28.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb05abbdf40 1 -- start start 2024-09-13T23:44:28.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb05abbdf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5632b9359400 0x5632b963e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb05abbdf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5632b941d500 con 0x5632b9359000 2024-09-13T23:44:28.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb05abbdf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5632b93e6a00 con 0x5632b9359400 2024-09-13T23:44:28.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb058b75700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5632b9359400 0x5632b963e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb059376700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5632b9359000 0x5632b9458800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:36362/0 (socket says 172.21.15.167:36362) 2024-09-13T23:44:28.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb058b75700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5632b9359400 0x5632b963e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42412/0 (socket says 172.21.15.167:42412) 2024-09-13T23:44:28.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb059376700 1 -- 172.21.15.167:0/3980422777 learned_addr learned my addr 172.21.15.167:0/3980422777 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:44:28.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb058b75700 1 -- 172.21.15.167:0/3980422777 >> v1:172.21.15.12:6789/0 conn(0x5632b9359000 legacy=0x5632b9458800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb058b75700 1 -- 172.21.15.167:0/3980422777 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632b936bd40 con 0x5632b9359400 2024-09-13T23:44:28.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb058b75700 1 --2- 172.21.15.167:0/3980422777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359400 0x5632b963e580 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x5632b9639920 tx=0x5632b945d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9275d7a0a3f559 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb058374700 1 -- 172.21.15.167:0/3980422777 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b93fd4a0 con 0x5632b9359400 2024-09-13T23:44:28.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb058374700 1 -- 172.21.15.167:0/3980422777 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632b941d800 con 0x5632b9359400 2024-09-13T23:44:28.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb058374700 1 -- 172.21.15.167:0/3980422777 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b93fd600 con 0x5632b9359400 2024-09-13T23:44:28.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3980422777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359400 msgr2=0x5632b963e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/3980422777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359400 0x5632b963e580 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x5632b9639920 tx=0x5632b945d920 comp rx=0 tx=0).stop 2024-09-13T23:44:28.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3980422777 shutdown_connections 2024-09-13T23:44:28.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/3980422777 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359400 0x5632b963e580 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3980422777 >> 172.21.15.167:0/3980422777 conn(0x5632b9358c00 msgr2=0x5632b9458000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3980422777 shutdown_connections 2024-09-13T23:44:28.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.789+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3980422777 wait complete. 2024-09-13T23:44:28.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 Processor -- start 2024-09-13T23:44:28.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 -- start start 2024-09-13T23:44:28.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 0x5632b963e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b93e6500 con 0x5632b9359000 2024-09-13T23:44:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb059376700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 0x5632b963e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb059376700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 0x5632b963e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42414/0 (socket says 172.21.15.167:42414) 2024-09-13T23:44:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb059376700 1 -- 172.21.15.167:0/2496772371 learned_addr learned my addr 172.21.15.167:0/2496772371 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb059376700 1 -- 172.21.15.167:0/2496772371 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632b936bba0 con 0x5632b9359000 2024-09-13T23:44:28.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb059376700 1 --2- 172.21.15.167:0/2496772371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 0x5632b963e580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x5632b94b5e90 tx=0x5632b945db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1541c9dccf7e2c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb057b73700 1 -- 172.21.15.167:0/2496772371 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b93fd1e0 con 0x5632b9359000 2024-09-13T23:44:28.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb057b73700 1 -- 172.21.15.167:0/2496772371 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632b941db00 con 0x5632b9359000 2024-09-13T23:44:28.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb057b73700 1 -- 172.21.15.167:0/2496772371 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b93fd4a0 con 0x5632b9359000 2024-09-13T23:44:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2496772371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 msgr2=0x5632b963e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/2496772371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 0x5632b963e580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x5632b94b5e90 tx=0x5632b945db60 comp rx=0 tx=0).stop 2024-09-13T23:44:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2496772371 shutdown_connections 2024-09-13T23:44:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/2496772371 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 0x5632b963e580 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2496772371 >> 172.21.15.167:0/2496772371 conn(0x5632b9358c00 msgr2=0x5632b9459000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2496772371 shutdown_connections 2024-09-13T23:44:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2496772371 wait complete. 2024-09-13T23:44:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 Processor -- start 2024-09-13T23:44:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 -- start start 2024-09-13T23:44:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 0x5632b963e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb05abbdf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b93e63c0 con 0x5632b9359000 2024-09-13T23:44:28.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb059376700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 0x5632b963e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb059376700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 0x5632b963e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42428/0 (socket says 172.21.15.167:42428) 2024-09-13T23:44:28.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.793+0000 7fb059376700 1 -- 172.21.15.167:0/3324035792 learned_addr learned my addr 172.21.15.167:0/3324035792 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb059376700 1 -- 172.21.15.167:0/3324035792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632b936ba00 con 0x5632b9359000 2024-09-13T23:44:28.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb059376700 1 --2- 172.21.15.167:0/3324035792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 0x5632b963e580 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x5632ba1cc0c0 tx=0x5632b945da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb056370700 1 -- 172.21.15.167:0/3324035792 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b93fd600 con 0x5632b9359000 2024-09-13T23:44:28.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb056370700 1 -- 172.21.15.167:0/3324035792 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632b941dc80 con 0x5632b9359000 2024-09-13T23:44:28.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb056370700 1 -- 172.21.15.167:0/3324035792 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b93fd1e0 con 0x5632b9359000 2024-09-13T23:44:28.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3324035792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5632b936b860 con 0x5632b9359000 2024-09-13T23:44:28.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3324035792 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632b936b6c0 con 0x5632b9359000 2024-09-13T23:44:28.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb05abbdf40 1 Processor -- start 2024-09-13T23:44:28.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb05abbdf40 1 -- start start 2024-09-13T23:44:28.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb05abbdf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 0x5632b963eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.803 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb05abbdf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b93e6000 con 0x5632b9359800 2024-09-13T23:44:28.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb056370700 1 -- 172.21.15.167:0/3324035792 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5632b9452b00 con 0x5632b9359000 2024-09-13T23:44:28.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb059b77700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 0x5632b963eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb059b77700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 0x5632b963eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42436/0 (socket says 172.21.15.167:42436) 2024-09-13T23:44:28.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb059b77700 1 -- 172.21.15.167:0/2086287637 learned_addr learned my addr 172.21.15.167:0/2086287637 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb056370700 1 --2- 172.21.15.167:0/3324035792 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9712000 0x5632b963f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.797+0000 7fb056370700 1 -- 172.21.15.167:0/3324035792 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5632b93bf180 con 0x5632b9359000 2024-09-13T23:44:28.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb059b77700 1 -- 172.21.15.167:0/2086287637 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632b936a000 con 0x5632b9359800 2024-09-13T23:44:28.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb059376700 1 --2- 172.21.15.167:0/3324035792 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9712000 0x5632b963f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb059b77700 1 --2- 172.21.15.167:0/2086287637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 0x5632b963eb00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x5632b96ec270 tx=0x5632b96c6120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82c499ee679b9a79 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb053b6b700 1 -- 172.21.15.167:0/2086287637 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b93fd600 con 0x5632b9359800 2024-09-13T23:44:28.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb053b6b700 1 -- 172.21.15.167:0/2086287637 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632b970a000 con 0x5632b9359800 2024-09-13T23:44:28.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb053b6b700 1 -- 172.21.15.167:0/2086287637 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b93fd1e0 con 0x5632b9359800 2024-09-13T23:44:28.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb059376700 1 --2- 172.21.15.167:0/3324035792 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9712000 0x5632b963f080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5632ba1cd680 tx=0x5632b96c6240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2086287637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 msgr2=0x5632b963eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/2086287637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 0x5632b963eb00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x5632b96ec270 tx=0x5632b96c6120 comp rx=0 tx=0).stop 2024-09-13T23:44:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2086287637 shutdown_connections 2024-09-13T23:44:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/2086287637 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 0x5632b963eb00 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2086287637 >> 172.21.15.167:0/2086287637 conn(0x5632b9359400 msgr2=0x5632b9459800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2086287637 shutdown_connections 2024-09-13T23:44:28.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2086287637 wait complete. 2024-09-13T23:44:28.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb05abbdf40 1 Processor -- start 2024-09-13T23:44:28.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb05abbdf40 1 -- start start 2024-09-13T23:44:28.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb05abbdf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 0x5632b963eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb05abbdf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b937e640 con 0x5632b9359800 2024-09-13T23:44:28.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb059b77700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 0x5632b963eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb059b77700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 0x5632b963eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42438/0 (socket says 172.21.15.167:42438) 2024-09-13T23:44:28.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.801+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 learned_addr learned my addr 172.21.15.167:0/1316982138 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632b936aea0 con 0x5632b9359800 2024-09-13T23:44:28.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb059b77700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 0x5632b963eb00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x5632b9638de0 tx=0x5632b96c68a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb052368700 1 -- 172.21.15.167:0/1316982138 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b93fda20 con 0x5632b9359800 2024-09-13T23:44:28.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb052368700 1 -- 172.21.15.167:0/1316982138 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632b970a180 con 0x5632b9359800 2024-09-13T23:44:28.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb052368700 1 -- 172.21.15.167:0/1316982138 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b93fd600 con 0x5632b9359800 2024-09-13T23:44:28.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5632b936a820 con 0x5632b9359800 2024-09-13T23:44:28.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632b936a9c0 con 0x5632b9359800 2024-09-13T23:44:28.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb05abbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb052368700 1 -- 172.21.15.167:0/1316982138 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5632b9452dc0 con 0x5632b9359800 2024-09-13T23:44:28.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb052368700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9712c00 0x5632b963f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb052368700 1 -- 172.21.15.167:0/1316982138 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5632b93bf340 con 0x5632b9359800 2024-09-13T23:44:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb058b75700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9712c00 0x5632b963f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5632b9713000 0x5632b963fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713400 con 0x5632b9713000 2024-09-13T23:44:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.805+0000 7fb059b77700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5632b9713000 0x5632b963fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb058b75700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9712c00 0x5632b963f600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5632b9639920 tx=0x5632b945d800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb059b77700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5632b9713000 0x5632b963fb80 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb05abbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb05abbdf40 20 realm 2024-09-13T23:44:28.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb05abbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713000 2024-09-13T23:44:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb05abbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb05abbdf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb05abbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713c00 con 0x5632b9713000 2024-09-13T23:44:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb05abbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb05abbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5632b9713400 0x5632b9640100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713400 2024-09-13T23:44:28.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.809+0000 7fb059376700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5632b9713400 0x5632b9640100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb059376700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5632b9713400 0x5632b9640100 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb05abbdf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:28.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb05abbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713c00 con 0x5632b9713400 2024-09-13T23:44:28.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb05abbdf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:28.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb05abbdf40 20 searching for the correct realm 2024-09-13T23:44:28.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5632b969a000 0x5632b9640680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969a400 con 0x5632b969a000 2024-09-13T23:44:28.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb058b75700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5632b969a000 0x5632b9640680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb058b75700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5632b969a000 0x5632b9640680 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.819 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.813+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969a400 con 0x5632b9713400 2024-09-13T23:44:28.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059376700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5632b969a400 0x5632b9640c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a400 2024-09-13T23:44:28.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059b77700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5632b969a400 0x5632b9640c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059b77700 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5632b969a400 0x5632b9640c00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713000 2024-09-13T23:44:28.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.822 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713000 2024-09-13T23:44:28.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a000 2024-09-13T23:44:28.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713000 2024-09-13T23:44:28.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.817+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713400 2024-09-13T23:44:28.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.821+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.821+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713400 2024-09-13T23:44:28.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.821+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.821+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713000 2024-09-13T23:44:28.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.821+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.821+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a400 2024-09-13T23:44:28.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.821+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.821+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713000 2024-09-13T23:44:28.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.821+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.821+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a400 2024-09-13T23:44:28.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.821+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.821+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713400 2024-09-13T23:44:28.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a000 2024-09-13T23:44:28.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713000 2024-09-13T23:44:28.829 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713400 2024-09-13T23:44:28.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a000 2024-09-13T23:44:28.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713000 2024-09-13T23:44:28.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a400 2024-09-13T23:44:28.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.825+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713000 2024-09-13T23:44:28.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.832 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a000 2024-09-13T23:44:28.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a400 2024-09-13T23:44:28.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713000 2024-09-13T23:44:28.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.834 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a400 2024-09-13T23:44:28.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a000 2024-09-13T23:44:28.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a400 2024-09-13T23:44:28.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.829+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713400 2024-09-13T23:44:28.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713400 2024-09-13T23:44:28.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a000 2024-09-13T23:44:28.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713400 2024-09-13T23:44:28.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713400 2024-09-13T23:44:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb05abbdf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb05abbdf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb05abbdf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb05abbdf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb05abbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.833+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969a800 con 0x5632b9713000 2024-09-13T23:44:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969ac00 con 0x5632b9713400 2024-09-13T23:44:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:28.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b000 con 0x5632b9713400 2024-09-13T23:44:28.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:28.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 zone default found 2024-09-13T23:44:28.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 4 Realm: () 2024-09-13T23:44:28.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:28.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:28.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:28.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 zonegroup default 2024-09-13T23:44:28.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b400 con 0x5632b969a400 2024-09-13T23:44:28.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b800 con 0x5632b969a000 2024-09-13T23:44:28.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969bc00 con 0x5632b969a000 2024-09-13T23:44:28.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:28.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 started sync module instance, tier type = 2024-09-13T23:44:28.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:28.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.837+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b800 con 0x5632b9713400 2024-09-13T23:44:28.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.841+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.841+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94775858649856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969bc00 con 0x5632b9713400 2024-09-13T23:44:28.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.841+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969ac00 con 0x5632b9713000 2024-09-13T23:44:28.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.845+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94775858649856] v35'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.845+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.850 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.845+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94775858651008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b000 con 0x5632b9713000 2024-09-13T23:44:28.851 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.845+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b800 con 0x5632b9713400 2024-09-13T23:44:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.849+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94775858651008] v35'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.849+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.849+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94775858652160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713c00 con 0x5632b9713400 2024-09-13T23:44:28.853 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.849+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a400 2024-09-13T23:44:28.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.849+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.849+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94775858653312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969a800 con 0x5632b969a400 2024-09-13T23:44:28.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.849+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b400 con 0x5632b9713400 2024-09-13T23:44:28.856 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.853+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94775858652160] v35'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.858 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.853+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94775858653312] v35'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.853+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.853+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94775858654464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969ac00 con 0x5632b9713400 2024-09-13T23:44:28.859 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.853+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b000 con 0x5632b969a000 2024-09-13T23:44:28.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.857+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.857+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94775858655616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d8000 con 0x5632b969a000 2024-09-13T23:44:28.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.857+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d8400 con 0x5632b969a000 2024-09-13T23:44:28.861 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.857+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94775858654464] v35'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.857+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94775858655616] v35'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.857+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.863 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.857+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94775858656768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d8800 con 0x5632b969a000 2024-09-13T23:44:28.864 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.857+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d8c00 con 0x5632b969a400 2024-09-13T23:44:28.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'192 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94775858657920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d9000 con 0x5632b969a400 2024-09-13T23:44:28.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb05abbdf40 20 add_watcher() i=0 2024-09-13T23:44:28.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb05abbdf40 20 add_watcher() i=1 2024-09-13T23:44:28.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb05abbdf40 20 add_watcher() i=2 2024-09-13T23:44:28.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb05abbdf40 20 add_watcher() i=3 2024-09-13T23:44:28.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb05abbdf40 20 add_watcher() i=4 2024-09-13T23:44:28.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb05abbdf40 20 add_watcher() i=5 2024-09-13T23:44:28.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94775858656768] v35'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb05abbdf40 20 add_watcher() i=6 2024-09-13T23:44:28.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94775858657920] v35'193 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb05abbdf40 20 add_watcher() i=7 2024-09-13T23:44:28.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb05abbdf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:28.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d9400 con 0x5632b9713000 2024-09-13T23:44:28.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.861+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94775860387840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d9800 con 0x5632b9713000 2024-09-13T23:44:28.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.865+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94775860387840] v35'198 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.865+0000 7fb040344700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:28.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.865+0000 7fb05abbdf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:28.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.865+0000 7fb05abbdf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:28.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.865+0000 7fb05abbdf40 1 Processor -- start 2024-09-13T23:44:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.865+0000 7fb05abbdf40 1 -- start start 2024-09-13T23:44:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.865+0000 7fb05abbdf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 0x5632b9641180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.865+0000 7fb05abbdf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b937e780 con 0x5632b97d8c00 2024-09-13T23:44:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.865+0000 7fb058b75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 0x5632b9641180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.865+0000 7fb058b75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 0x5632b9641180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42442/0 (socket says 172.21.15.167:42442) 2024-09-13T23:44:28.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.865+0000 7fb058b75700 1 -- 172.21.15.167:0/4133242205 learned_addr learned my addr 172.21.15.167:0/4133242205 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.865+0000 7fb058b75700 1 -- 172.21.15.167:0/4133242205 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632b936ab60 con 0x5632b97d8c00 2024-09-13T23:44:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb058b75700 1 --2- 172.21.15.167:0/4133242205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 0x5632b9641180 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x5632b977bc20 tx=0x5632b945d860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71aae4304e2c845c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb03fb43700 1 -- 172.21.15.167:0/4133242205 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b93fd760 con 0x5632b97d8c00 2024-09-13T23:44:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb03fb43700 1 -- 172.21.15.167:0/4133242205 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632b970a600 con 0x5632b97d8c00 2024-09-13T23:44:28.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb03fb43700 1 -- 172.21.15.167:0/4133242205 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b97f0000 con 0x5632b97d8c00 2024-09-13T23:44:28.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb05abbdf40 1 -- 172.21.15.167:0/4133242205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 msgr2=0x5632b9641180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/4133242205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 0x5632b9641180 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x5632b977bc20 tx=0x5632b945d860 comp rx=0 tx=0).stop 2024-09-13T23:44:28.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb05abbdf40 1 -- 172.21.15.167:0/4133242205 shutdown_connections 2024-09-13T23:44:28.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/4133242205 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 0x5632b9641180 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb05abbdf40 1 -- 172.21.15.167:0/4133242205 >> 172.21.15.167:0/4133242205 conn(0x5632b97d9c00 msgr2=0x5632b945a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb05abbdf40 1 -- 172.21.15.167:0/4133242205 shutdown_connections 2024-09-13T23:44:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb05abbdf40 1 -- 172.21.15.167:0/4133242205 wait complete. 2024-09-13T23:44:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb05abbdf40 1 Processor -- start 2024-09-13T23:44:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb05abbdf40 1 -- start start 2024-09-13T23:44:28.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb05abbdf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 0x5632b9641180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb05abbdf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632b93e6000 con 0x5632b97d8c00 2024-09-13T23:44:28.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb058b75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 0x5632b9641180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb058b75700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 0x5632b9641180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42446/0 (socket says 172.21.15.167:42446) 2024-09-13T23:44:28.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb058b75700 1 -- 172.21.15.167:0/2971821328 learned_addr learned my addr 172.21.15.167:0/2971821328 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb058b75700 1 -- 172.21.15.167:0/2971821328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632b936ad00 con 0x5632b97d8c00 2024-09-13T23:44:28.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.869+0000 7fb058b75700 1 --2- 172.21.15.167:0/2971821328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 0x5632b9641180 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x5632b97e81e0 tx=0x5632b97b6420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.873+0000 7fb03e340700 1 -- 172.21.15.167:0/2971821328 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b97f0160 con 0x5632b97d8c00 2024-09-13T23:44:28.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.873+0000 7fb03e340700 1 -- 172.21.15.167:0/2971821328 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5632b970a300 con 0x5632b97d8c00 2024-09-13T23:44:28.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.873+0000 7fb03e340700 1 -- 172.21.15.167:0/2971821328 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5632b97f02c0 con 0x5632b97d8c00 2024-09-13T23:44:28.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.873+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2971821328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5632b936a340 con 0x5632b97d8c00 2024-09-13T23:44:28.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.873+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2971821328 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632b936bd40 con 0x5632b97d8c00 2024-09-13T23:44:28.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.873+0000 7fb03e340700 1 -- 172.21.15.167:0/2971821328 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5632b94538c0 con 0x5632b97d8c00 2024-09-13T23:44:28.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.873+0000 7fb03e340700 1 --2- 172.21.15.167:0/2971821328 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9860000 0x5632b9866000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.873+0000 7fb03e340700 1 -- 172.21.15.167:0/2971821328 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5632b93bf500 con 0x5632b97d8c00 2024-09-13T23:44:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.873+0000 7fb05abbdf40 5 note: GC not initialized 2024-09-13T23:44:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.873+0000 7fb059b77700 1 --2- 172.21.15.167:0/2971821328 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9860000 0x5632b9866000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.873+0000 7fb059b77700 1 --2- 172.21.15.167:0/2971821328 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9860000 0x5632b9866000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x5632b977b230 tx=0x5632b96c77a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb03bb3b700 20 reqs_thread_entry: start 2024-09-13T23:44:28.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb05abbdf40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:28.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb05abbdf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:28.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb039b37700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:59 2024 2024-09-13T23:44:28.882 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:28.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d8000 con 0x5632b9713400 2024-09-13T23:44:28.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb039336700 20 reqs_thread_entry: start 2024-09-13T23:44:28.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d8400 con 0x5632b9713400 2024-09-13T23:44:28.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969a800 con 0x5632b9713400 2024-09-13T23:44:28.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b000 con 0x5632b969a000 2024-09-13T23:44:28.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.877+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b9713400 2024-09-13T23:44:28.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.881+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.881+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9860400 con 0x5632b969a000 2024-09-13T23:44:28.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.881+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.881+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9860800 con 0x5632b9713000 2024-09-13T23:44:28.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.881+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.881+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9860c00 con 0x5632b9713000 2024-09-13T23:44:28.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.881+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.881+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9861000 con 0x5632b969a000 2024-09-13T23:44:28.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.881+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.881+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9861400 con 0x5632b969a000 2024-09-13T23:44:28.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.881+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 151+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.881+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9861800 con 0x5632b969a000 2024-09-13T23:44:28.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(72 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9861c00 con 0x5632b9713000 2024-09-13T23:44:28.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d8000 con 0x5632b969a400 2024-09-13T23:44:28.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969bc00 con 0x5632b969a000 2024-09-13T23:44:28.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b800 con 0x5632b969a400 2024-09-13T23:44:28.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b400 con 0x5632b9713000 2024-09-13T23:44:28.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969ac00 con 0x5632b969a000 2024-09-13T23:44:28.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713c00 con 0x5632b9713000 2024-09-13T23:44:28.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(79 usage.17 [call out=163b] v0'0 uv5 ondisk = 0) v8 ==== 152+0+163 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.885+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9860800 con 0x5632b9713400 2024-09-13T23:44:28.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.889+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.889+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9860c00 con 0x5632b9713400 2024-09-13T23:44:28.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.889+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.889+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9861c00 con 0x5632b969a000 2024-09-13T23:44:28.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.889+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.889+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9860400 con 0x5632b9713400 2024-09-13T23:44:28.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.889+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.889+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9861000 con 0x5632b969a000 2024-09-13T23:44:28.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.889+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.889+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9861400 con 0x5632b9713000 2024-09-13T23:44:28.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.889+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.889+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b800 con 0x5632b9713400 2024-09-13T23:44:28.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.893+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.893+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b000 con 0x5632b9713400 2024-09-13T23:44:28.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.893+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.893+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d9800 con 0x5632b9713400 2024-09-13T23:44:28.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.893+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.893+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d8000 con 0x5632b9713000 2024-09-13T23:44:28.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.893+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.893+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9860800 con 0x5632b969a000 2024-09-13T23:44:28.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.893+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.893+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9860c00 con 0x5632b969a000 2024-09-13T23:44:28.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.893+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.897+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9860400 con 0x5632b9713400 2024-09-13T23:44:28.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.897+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.897+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9861800 con 0x5632b9713000 2024-09-13T23:44:28.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.897+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.902 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:44:28.902 INFO:teuthology.orchestra.run.smithi167.stdout: "entries": [ 2024-09-13T23:44:28.902 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.902 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "anonymous", 2024-09-13T23:44:28.902 INFO:teuthology.orchestra.run.smithi167.stdout: "buckets": [ 2024-09-13T23:44:28.902 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.902 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "", 2024-09-13T23:44:28.902 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:28.902 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "anonymous", 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "list_buckets", 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 231, 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.903 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: "buckets": [ 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "", 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "list_buckets", 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 322, 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.904 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo", 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.905 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 5, 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 5 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 236, 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 2, 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_obj", 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.906 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 4, 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "get_acls", 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 1026, 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 2, 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 2 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "get_obj", 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 4, 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.907 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.908 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.908 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.908 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "list_bucket", 2024-09-13T23:44:28.908 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 515, 2024-09-13T23:44:28.908 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.908 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.908 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.908 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.908 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.908 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "put_acls", 2024-09-13T23:44:28.908 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.908 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 2, 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 2 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "put_obj", 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 30000042, 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 11, 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 11 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.909 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo2", 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.910 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo3", 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:28.911 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.912 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:28.912 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.912 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.912 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.912 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.912 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.912 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.912 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:28.912 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.912 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.912 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.912 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo4", 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.913 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.914 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: "summary": [ 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "anonymous", 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "list_buckets", 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 231, 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: "total": { 2024-09-13T23:44:28.915 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 231, 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 8, 2024-09-13T23:44:28.916 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 8 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 236, 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 5, 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_obj", 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.917 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 4, 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "get_acls", 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 1026, 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 2, 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 2 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "get_obj", 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 4, 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.918 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.919 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.919 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.897+0000 7fb040344700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:28.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.897+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94775860387840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9861c00 con 0x5632b9713000 2024-09-13T23:44:28.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.901+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94775860387840] v35'199 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.901+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94775858649856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9861000 con 0x5632b9713400 2024-09-13T23:44:28.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.905+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 31 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94775858649856] v35'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.905+0000 7fb05abbdf40 20 remove_watcher() i=0 2024-09-13T23:44:28.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.905+0000 7fb05abbdf40 2 removed watcher, disabling cache 2024-09-13T23:44:28.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.905+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94775858651008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969a800 con 0x5632b9713000 2024-09-13T23:44:28.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.905+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94775858651008] v35'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b9713000 2024-09-13T23:44:28.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.905+0000 7fb05abbdf40 20 remove_watcher() i=1 2024-09-13T23:44:28.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.905+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94775858652160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b800 con 0x5632b9713400 2024-09-13T23:44:28.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.909+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 32 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94775858652160] v35'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.909+0000 7fb05abbdf40 20 remove_watcher() i=2 2024-09-13T23:44:28.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.909+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94775858653312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969bc00 con 0x5632b969a400 2024-09-13T23:44:28.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.913+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94775858653312] v35'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.913+0000 7fb05abbdf40 20 remove_watcher() i=3 2024-09-13T23:44:28.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.913+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94775858654464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969ac00 con 0x5632b9713400 2024-09-13T23:44:28.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.917+0000 7fb059376700 1 -- 172.21.15.167:0/1316982138 <== osd.0 v2:172.21.15.12:6800/2649399468 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94775858654464] v35'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777200 con 0x5632b9713400 2024-09-13T23:44:28.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.917+0000 7fb05abbdf40 20 remove_watcher() i=4 2024-09-13T23:44:28.922 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.917+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94775858655616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b9713800 con 0x5632b969a000 2024-09-13T23:44:28.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.917+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94775858655616] v35'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.917+0000 7fb05abbdf40 20 remove_watcher() i=5 2024-09-13T23:44:28.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.917+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94775858656768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b97d8400 con 0x5632b969a000 2024-09-13T23:44:28.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.921+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 <== osd.3 v2:172.21.15.167:6816/3688363524 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94775858656768] v35'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9777440 con 0x5632b969a000 2024-09-13T23:44:28.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.921+0000 7fb05abbdf40 20 remove_watcher() i=6 2024-09-13T23:44:28.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.921+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94775858657920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5632b969b400 con 0x5632b969a400 2024-09-13T23:44:28.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb059b77700 1 -- 172.21.15.167:0/1316982138 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94775858657920] v35'194 uv192 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5632b9776b40 con 0x5632b969a400 2024-09-13T23:44:28.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 20 remove_watcher() i=7 2024-09-13T23:44:28.929 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5632b9713400 msgr2=0x5632b9640100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5632b9713400 0x5632b9640100 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5632b9713000 msgr2=0x5632b963fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5632b9713000 0x5632b963fb80 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5632b969a400 msgr2=0x5632b9640c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5632b969a400 0x5632b9640c00 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5632b969a000 msgr2=0x5632b9640680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5632b969a000 0x5632b9640680 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9712c00 msgr2=0x5632b963f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9712c00 0x5632b963f600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x5632b9639920 tx=0x5632b945d800 comp rx=0 tx=0).stop 2024-09-13T23:44:28.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 msgr2=0x5632b963eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 0x5632b963eb00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x5632b9638de0 tx=0x5632b96c68a0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb058b75700 1 -- 172.21.15.167:0/1316982138 reap_dead start 2024-09-13T23:44:28.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 shutdown_connections 2024-09-13T23:44:28.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/1316982138 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359800 0x5632b963eb00 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 >> 172.21.15.167:0/1316982138 conn(0x5632b9359400 msgr2=0x5632b9459800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 shutdown_connections 2024-09-13T23:44:28.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/1316982138 wait complete. 2024-09-13T23:44:28.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2971821328 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9860000 msgr2=0x5632b9866000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/2971821328 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9860000 0x5632b9866000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x5632b977b230 tx=0x5632b96c77a0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2971821328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 msgr2=0x5632b9641180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/2971821328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 0x5632b9641180 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x5632b97e81e0 tx=0x5632b97b6420 comp rx=0 tx=0).stop 2024-09-13T23:44:28.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2971821328 shutdown_connections 2024-09-13T23:44:28.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/2971821328 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9860000 0x5632b9866000 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/2971821328 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b97d8c00 0x5632b9641180 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2971821328 >> 172.21.15.167:0/2971821328 conn(0x5632b97d9c00 msgr2=0x5632b945a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2971821328 shutdown_connections 2024-09-13T23:44:28.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/2971821328 wait complete. 2024-09-13T23:44:28.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3324035792 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9712000 msgr2=0x5632b963f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/3324035792 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9712000 0x5632b963f080 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x5632ba1cd680 tx=0x5632b96c6240 comp rx=0 tx=0).stop 2024-09-13T23:44:28.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3324035792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 msgr2=0x5632b963e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/3324035792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 0x5632b963e580 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x5632ba1cc0c0 tx=0x5632b945da40 comp rx=0 tx=0).stop 2024-09-13T23:44:28.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3324035792 shutdown_connections 2024-09-13T23:44:28.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/3324035792 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5632b9712000 0x5632b963f080 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 --2- 172.21.15.167:0/3324035792 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5632b9359000 0x5632b963e580 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3324035792 >> 172.21.15.167:0/3324035792 conn(0x5632b9358c00 msgr2=0x5632b9459000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3324035792 shutdown_connections 2024-09-13T23:44:28.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.925+0000 7fb05abbdf40 1 -- 172.21.15.167:0/3324035792 wait complete. 2024-09-13T23:44:28.934 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.934 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "list_bucket", 2024-09-13T23:44:28.934 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 515, 2024-09-13T23:44:28.934 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.934 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "list_buckets", 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 322, 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "put_acls", 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.935 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 2, 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 2 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "put_obj", 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 30000042, 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 11, 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 11 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: "total": { 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 2103, 2024-09-13T23:44:28.936 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 30000042, 2024-09-13T23:44:28.937 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 35, 2024-09-13T23:44:28.937 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 34 2024-09-13T23:44:28.937 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.937 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:28.937 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:28.937 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:44:28.938 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34}}]} 2024-09-13T23:44:28.938 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-09-13T23:44:28.938 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-09-13T23:44:28.939 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-09-13T23:44:28.980 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:28.980 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:28.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32f992f40 1 Processor -- start 2024-09-13T23:44:28.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32f992f40 1 -- start start 2024-09-13T23:44:28.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32f992f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56466fbe1400 0x56466fec6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32f992f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56466fca5500 con 0x56466fbe1000 2024-09-13T23:44:28.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32f992f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x56466fc6ea00 con 0x56466fbe1400 2024-09-13T23:44:28.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32d94a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56466fbe1400 0x56466fec6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32e14b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x56466fbe1000 0x56466fce0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:36372/0 (socket says 172.21.15.167:36372) 2024-09-13T23:44:28.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32e14b700 1 -- 172.21.15.167:0/2010725434 learned_addr learned my addr 172.21.15.167:0/2010725434 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:44:28.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32d94a700 1 -- 172.21.15.167:0/2010725434 >> v1:172.21.15.12:6789/0 conn(0x56466fbe1000 legacy=0x56466fce0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32d94a700 1 -- 172.21.15.167:0/2010725434 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56466fbf3d40 con 0x56466fbe1400 2024-09-13T23:44:28.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32d94a700 1 --2- 172.21.15.167:0/2010725434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1400 0x56466fec6580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x56466fec1920 tx=0x56466fce5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e416d8eda26a6a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32d149700 1 -- 172.21.15.167:0/2010725434 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56466fc854a0 con 0x56466fbe1400 2024-09-13T23:44:28.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32d149700 1 -- 172.21.15.167:0/2010725434 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56466fca5800 con 0x56466fbe1400 2024-09-13T23:44:28.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32d149700 1 -- 172.21.15.167:0/2010725434 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56466fc85600 con 0x56466fbe1400 2024-09-13T23:44:28.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32f992f40 1 -- 172.21.15.167:0/2010725434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1400 msgr2=0x56466fec6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32f992f40 1 --2- 172.21.15.167:0/2010725434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1400 0x56466fec6580 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x56466fec1920 tx=0x56466fce5920 comp rx=0 tx=0).stop 2024-09-13T23:44:28.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32f992f40 1 -- 172.21.15.167:0/2010725434 shutdown_connections 2024-09-13T23:44:28.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32f992f40 1 --2- 172.21.15.167:0/2010725434 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1400 0x56466fec6580 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32f992f40 1 -- 172.21.15.167:0/2010725434 >> 172.21.15.167:0/2010725434 conn(0x56466fbe0c00 msgr2=0x56466fce0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32f992f40 1 -- 172.21.15.167:0/2010725434 shutdown_connections 2024-09-13T23:44:28.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.977+0000 7fd32f992f40 1 -- 172.21.15.167:0/2010725434 wait complete. 2024-09-13T23:44:28.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 Processor -- start 2024-09-13T23:44:28.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 -- start start 2024-09-13T23:44:28.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 0x56466fec6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466fc6e500 con 0x56466fbe1000 2024-09-13T23:44:28.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32e14b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 0x56466fec6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32e14b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 0x56466fec6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42460/0 (socket says 172.21.15.167:42460) 2024-09-13T23:44:28.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32e14b700 1 -- 172.21.15.167:0/2875250015 learned_addr learned my addr 172.21.15.167:0/2875250015 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32e14b700 1 -- 172.21.15.167:0/2875250015 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56466fbf3ba0 con 0x56466fbe1000 2024-09-13T23:44:28.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32e14b700 1 --2- 172.21.15.167:0/2875250015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 0x56466fec6580 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x56466fd3de00 tx=0x56466fce5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d27de839464a030 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32c948700 1 -- 172.21.15.167:0/2875250015 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56466fc854a0 con 0x56466fbe1000 2024-09-13T23:44:28.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32c948700 1 -- 172.21.15.167:0/2875250015 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56466fca5b00 con 0x56466fbe1000 2024-09-13T23:44:28.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32c948700 1 -- 172.21.15.167:0/2875250015 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56466fc85600 con 0x56466fbe1000 2024-09-13T23:44:28.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 -- 172.21.15.167:0/2875250015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 msgr2=0x56466fec6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 --2- 172.21.15.167:0/2875250015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 0x56466fec6580 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x56466fd3de00 tx=0x56466fce5b60 comp rx=0 tx=0).stop 2024-09-13T23:44:28.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 -- 172.21.15.167:0/2875250015 shutdown_connections 2024-09-13T23:44:28.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 --2- 172.21.15.167:0/2875250015 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 0x56466fec6580 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 -- 172.21.15.167:0/2875250015 >> 172.21.15.167:0/2875250015 conn(0x56466fbe0c00 msgr2=0x56466fce1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 -- 172.21.15.167:0/2875250015 shutdown_connections 2024-09-13T23:44:28.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 -- 172.21.15.167:0/2875250015 wait complete. 2024-09-13T23:44:28.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 Processor -- start 2024-09-13T23:44:28.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 -- start start 2024-09-13T23:44:28.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 0x56466fec6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32f992f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466fc6e3c0 con 0x56466fbe1000 2024-09-13T23:44:28.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32e14b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 0x56466fec6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32e14b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 0x56466fec6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42472/0 (socket says 172.21.15.167:42472) 2024-09-13T23:44:28.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.981+0000 7fd32e14b700 1 -- 172.21.15.167:0/3572748319 learned_addr learned my addr 172.21.15.167:0/3572748319 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32e14b700 1 -- 172.21.15.167:0/3572748319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56466fbf3a00 con 0x56466fbe1000 2024-09-13T23:44:28.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32e14b700 1 --2- 172.21.15.167:0/3572748319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 0x56466fec6580 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x56466fd3def0 tx=0x56466fce5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32b145700 1 -- 172.21.15.167:0/3572748319 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56466fc858c0 con 0x56466fbe1000 2024-09-13T23:44:28.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32b145700 1 -- 172.21.15.167:0/3572748319 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56466fca5c80 con 0x56466fbe1000 2024-09-13T23:44:28.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32b145700 1 -- 172.21.15.167:0/3572748319 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56466fc854a0 con 0x56466fbe1000 2024-09-13T23:44:28.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32f992f40 1 -- 172.21.15.167:0/3572748319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56466fbf3860 con 0x56466fbe1000 2024-09-13T23:44:28.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32f992f40 1 -- 172.21.15.167:0/3572748319 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56466fbf36c0 con 0x56466fbe1000 2024-09-13T23:44:28.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32f992f40 1 Processor -- start 2024-09-13T23:44:28.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32f992f40 1 -- start start 2024-09-13T23:44:28.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32b145700 1 -- 172.21.15.167:0/3572748319 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56466fcdab00 con 0x56466fbe1000 2024-09-13T23:44:28.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32f992f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 0x56466fec6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32f992f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466fc6e000 con 0x56466fbe1800 2024-09-13T23:44:28.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32b145700 1 --2- 172.21.15.167:0/3572748319 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56466ff8c000 0x56466fec7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32e94c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 0x56466fec6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32e94c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 0x56466fec6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42478/0 (socket says 172.21.15.167:42478) 2024-09-13T23:44:28.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32e94c700 1 -- 172.21.15.167:0/2932772294 learned_addr learned my addr 172.21.15.167:0/2932772294 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32b145700 1 -- 172.21.15.167:0/3572748319 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x56466fc47180 con 0x56466fbe1000 2024-09-13T23:44:28.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32e14b700 1 --2- 172.21.15.167:0/3572748319 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56466ff8c000 0x56466fec7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.985+0000 7fd32e94c700 1 -- 172.21.15.167:0/2932772294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56466fbf2000 con 0x56466fbe1800 2024-09-13T23:44:28.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32e94c700 1 --2- 172.21.15.167:0/2932772294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 0x56466fec6b00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x56466fd3c030 tx=0x56466ff44de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf2444f8d0127e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32e14b700 1 --2- 172.21.15.167:0/3572748319 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56466ff8c000 0x56466fec7080 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x56466ff10f60 tx=0x56466ff44240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd328940700 1 -- 172.21.15.167:0/2932772294 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56466fc858c0 con 0x56466fbe1800 2024-09-13T23:44:28.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd328940700 1 -- 172.21.15.167:0/2932772294 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56466ff98000 con 0x56466fbe1800 2024-09-13T23:44:28.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd328940700 1 -- 172.21.15.167:0/2932772294 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56466fc854a0 con 0x56466fbe1800 2024-09-13T23:44:28.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32f992f40 1 -- 172.21.15.167:0/2932772294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 msgr2=0x56466fec6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:28.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32f992f40 1 --2- 172.21.15.167:0/2932772294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 0x56466fec6b00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x56466fd3c030 tx=0x56466ff44de0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32f992f40 1 -- 172.21.15.167:0/2932772294 shutdown_connections 2024-09-13T23:44:28.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32f992f40 1 --2- 172.21.15.167:0/2932772294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 0x56466fec6b00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32f992f40 1 -- 172.21.15.167:0/2932772294 >> 172.21.15.167:0/2932772294 conn(0x56466fbe1400 msgr2=0x56466fce1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32f992f40 1 -- 172.21.15.167:0/2932772294 shutdown_connections 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32f992f40 1 -- 172.21.15.167:0/2932772294 wait complete. 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32f992f40 1 Processor -- start 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32f992f40 1 -- start start 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32f992f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 0x56466fec6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32f992f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466fc06640 con 0x56466fbe1800 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32e94c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 0x56466fec6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32e94c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 0x56466fec6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42488/0 (socket says 172.21.15.167:42488) 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 learned_addr learned my addr 172.21.15.167:0/3433621883 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56466fbf2ea0 con 0x56466fbe1800 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32e94c700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 0x56466fec6b00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x56466ff10840 tx=0x56466ff44840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:28.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32713d700 1 -- 172.21.15.167:0/3433621883 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56466fc85b80 con 0x56466fbe1800 2024-09-13T23:44:28.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32713d700 1 -- 172.21.15.167:0/3433621883 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56466ff98180 con 0x56466fbe1800 2024-09-13T23:44:28.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56466fbf2820 con 0x56466fbe1800 2024-09-13T23:44:28.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.989+0000 7fd32713d700 1 -- 172.21.15.167:0/3433621883 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56466fc858c0 con 0x56466fbe1800 2024-09-13T23:44:28.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.993+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56466fbf29c0 con 0x56466fbe1800 2024-09-13T23:44:28.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.993+0000 7fd32f992f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:28.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.993+0000 7fd32713d700 1 -- 172.21.15.167:0/3433621883 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56466fcdadc0 con 0x56466fbe1800 2024-09-13T23:44:28.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.993+0000 7fd32713d700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56466ff8cc00 0x56466fec7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:28.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.993+0000 7fd32713d700 1 -- 172.21.15.167:0/3433621883 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x56466fc47340 con 0x56466fbe1800 2024-09-13T23:44:28.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.993+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56466ff8d000 0x56466fec7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.993+0000 7fd32d94a700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56466ff8cc00 0x56466fec7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.993+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d400 con 0x56466ff8d000 2024-09-13T23:44:29.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.993+0000 7fd32e94c700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56466ff8d000 0x56466fec7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.993+0000 7fd32d94a700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56466ff8cc00 0x56466fec7600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x56466fec1920 tx=0x56466fce5800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.993+0000 7fd32e94c700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56466ff8d000 0x56466fec7b80 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 20 realm 2024-09-13T23:44:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d000 2024-09-13T23:44:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:29.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8dc00 con 0x56466ff8d000 2024-09-13T23:44:29.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56466ff8d400 0x56466fec8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d400 2024-09-13T23:44:29.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32e14b700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56466ff8d400 0x56466fec8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32e14b700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56466ff8d400 0x56466fec8100 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8dc00 con 0x56466ff8d400 2024-09-13T23:44:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:28.997+0000 7fd32f992f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:29.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32f992f40 20 searching for the correct realm 2024-09-13T23:44:29.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56466ff2a000 0x56466fec8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2a400 con 0x56466ff2a000 2024-09-13T23:44:29.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32d94a700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56466ff2a000 0x56466fec8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32d94a700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56466ff2a000 0x56466fec8680 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2a400 con 0x56466ff8d400 2024-09-13T23:44:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32e14b700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56466ff2a400 0x56466fec8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a400 2024-09-13T23:44:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32e94c700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56466ff2a400 0x56466fec8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32e94c700 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56466ff2a400 0x56466fec8c00 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.001+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d000 2024-09-13T23:44:29.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d000 2024-09-13T23:44:29.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a000 2024-09-13T23:44:29.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d000 2024-09-13T23:44:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d400 2024-09-13T23:44:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d400 2024-09-13T23:44:29.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d000 2024-09-13T23:44:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.005+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a400 2024-09-13T23:44:29.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d000 2024-09-13T23:44:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a400 2024-09-13T23:44:29.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d400 2024-09-13T23:44:29.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a000 2024-09-13T23:44:29.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d000 2024-09-13T23:44:29.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d400 2024-09-13T23:44:29.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.009+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a000 2024-09-13T23:44:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d000 2024-09-13T23:44:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a400 2024-09-13T23:44:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d000 2024-09-13T23:44:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a000 2024-09-13T23:44:29.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a400 2024-09-13T23:44:29.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d000 2024-09-13T23:44:29.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.013+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a400 2024-09-13T23:44:29.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.017+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.017+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a000 2024-09-13T23:44:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.017+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.017+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a400 2024-09-13T23:44:29.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.017+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.017+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d400 2024-09-13T23:44:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.017+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.017+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d400 2024-09-13T23:44:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.017+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.017+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a000 2024-09-13T23:44:29.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.017+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.017+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d400 2024-09-13T23:44:29.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d400 2024-09-13T23:44:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32f992f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32f992f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32f992f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32f992f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32f992f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2a800 con 0x56466ff8d000 2024-09-13T23:44:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32f992f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32f992f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2ac00 con 0x56466ff8d400 2024-09-13T23:44:29.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32f992f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:29.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32f992f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2b000 con 0x56466ff8d400 2024-09-13T23:44:29.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.021+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 20 zone default found 2024-09-13T23:44:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 4 Realm: () 2024-09-13T23:44:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 20 zonegroup default 2024-09-13T23:44:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2b400 con 0x56466ff2a400 2024-09-13T23:44:29.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2b800 con 0x56466ff2a000 2024-09-13T23:44:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2bc00 con 0x56466ff2a000 2024-09-13T23:44:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 20 started sync module instance, tier type = 2024-09-13T23:44:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:29.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.025+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2b800 con 0x56466ff8d400 2024-09-13T23:44:29.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.029+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.029+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94860525427456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2bc00 con 0x56466ff8d400 2024-09-13T23:44:29.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.029+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2ac00 con 0x56466ff8d000 2024-09-13T23:44:29.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.033+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.033+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94860525428608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2b000 con 0x56466ff8d000 2024-09-13T23:44:29.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.033+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2b800 con 0x56466ff8d400 2024-09-13T23:44:29.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.033+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94860525427456] v35'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.033+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94860525428608] v35'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.033+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.033+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94860525429760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8dc00 con 0x56466ff8d400 2024-09-13T23:44:29.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.033+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff2a400 2024-09-13T23:44:29.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.037+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.037+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94860525430912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2a800 con 0x56466ff2a400 2024-09-13T23:44:29.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.037+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2b400 con 0x56466ff8d400 2024-09-13T23:44:29.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.037+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94860525429760] v35'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.041+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94860525430912] v35'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.041+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.041+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94860525432064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2ac00 con 0x56466ff8d400 2024-09-13T23:44:29.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.041+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2b000 con 0x56466ff2a000 2024-09-13T23:44:29.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.041+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.041+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94860525433216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564670056000 con 0x56466ff2a000 2024-09-13T23:44:29.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.041+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94860525432064] v35'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.041+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564670056400 con 0x56466ff2a000 2024-09-13T23:44:29.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.045+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94860525433216] v35'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.045+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.045+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94860525434368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564670056800 con 0x56466ff2a000 2024-09-13T23:44:29.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.045+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564670056c00 con 0x56466ff2a400 2024-09-13T23:44:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94860525434368] v35'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'195 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94860525435520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564670057000 con 0x56466ff2a400 2024-09-13T23:44:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32f992f40 20 add_watcher() i=0 2024-09-13T23:44:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32f992f40 20 add_watcher() i=1 2024-09-13T23:44:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32f992f40 20 add_watcher() i=2 2024-09-13T23:44:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32f992f40 20 add_watcher() i=3 2024-09-13T23:44:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32f992f40 20 add_watcher() i=4 2024-09-13T23:44:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32f992f40 20 add_watcher() i=5 2024-09-13T23:44:29.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32f992f40 20 add_watcher() i=6 2024-09-13T23:44:29.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94860525435520] v35'196 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32f992f40 20 add_watcher() i=7 2024-09-13T23:44:29.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32f992f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:29.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564670057400 con 0x56466ff8d000 2024-09-13T23:44:29.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.049+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94860527124480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564670057800 con 0x56466ff8d000 2024-09-13T23:44:29.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94860527124480] v35'200 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.057 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd32f992f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:29.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd32f992f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:29.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd32f992f40 1 Processor -- start 2024-09-13T23:44:29.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd315119700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:29.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd32f992f40 1 -- start start 2024-09-13T23:44:29.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd32f992f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 0x56466fec9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd32f992f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466fc06780 con 0x564670056c00 2024-09-13T23:44:29.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd32d94a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 0x56466fec9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd32d94a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 0x56466fec9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42502/0 (socket says 172.21.15.167:42502) 2024-09-13T23:44:29.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd32d94a700 1 -- 172.21.15.167:0/4149737199 learned_addr learned my addr 172.21.15.167:0/4149737199 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd32d94a700 1 -- 172.21.15.167:0/4149737199 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56466fbf2b60 con 0x564670056c00 2024-09-13T23:44:29.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd32d94a700 1 --2- 172.21.15.167:0/4149737199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 0x56466fec9180 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x5646700640f0 tx=0x56466fce5860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a03881ad3ac8bbc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.053+0000 7fd314918700 1 -- 172.21.15.167:0/4149737199 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56466fc85760 con 0x564670056c00 2024-09-13T23:44:29.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd314918700 1 -- 172.21.15.167:0/4149737199 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56466ff98600 con 0x564670056c00 2024-09-13T23:44:29.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd314918700 1 -- 172.21.15.167:0/4149737199 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56467006e000 con 0x564670056c00 2024-09-13T23:44:29.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32f992f40 1 -- 172.21.15.167:0/4149737199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 msgr2=0x56466fec9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32f992f40 1 --2- 172.21.15.167:0/4149737199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 0x56466fec9180 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x5646700640f0 tx=0x56466fce5860 comp rx=0 tx=0).stop 2024-09-13T23:44:29.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32f992f40 1 -- 172.21.15.167:0/4149737199 shutdown_connections 2024-09-13T23:44:29.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32f992f40 1 --2- 172.21.15.167:0/4149737199 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 0x56466fec9180 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32f992f40 1 -- 172.21.15.167:0/4149737199 >> 172.21.15.167:0/4149737199 conn(0x564670057c00 msgr2=0x56466fce2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32f992f40 1 -- 172.21.15.167:0/4149737199 shutdown_connections 2024-09-13T23:44:29.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32f992f40 1 -- 172.21.15.167:0/4149737199 wait complete. 2024-09-13T23:44:29.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32f992f40 1 Processor -- start 2024-09-13T23:44:29.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32f992f40 1 -- start start 2024-09-13T23:44:29.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32f992f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 0x56466fec9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32f992f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56466fc6e000 con 0x564670056c00 2024-09-13T23:44:29.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32d94a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 0x56466fec9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32d94a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 0x56466fec9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42508/0 (socket says 172.21.15.167:42508) 2024-09-13T23:44:29.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32d94a700 1 -- 172.21.15.167:0/1567673027 learned_addr learned my addr 172.21.15.167:0/1567673027 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32d94a700 1 -- 172.21.15.167:0/1567673027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56466fbf2d00 con 0x564670056c00 2024-09-13T23:44:29.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32d94a700 1 --2- 172.21.15.167:0/1567673027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 0x56466fec9180 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x564670064390 tx=0x564670036420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd313115700 1 -- 172.21.15.167:0/1567673027 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56467006e160 con 0x564670056c00 2024-09-13T23:44:29.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd313115700 1 -- 172.21.15.167:0/1567673027 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56466ff98300 con 0x564670056c00 2024-09-13T23:44:29.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd32f992f40 1 -- 172.21.15.167:0/1567673027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56466fbf2340 con 0x564670056c00 2024-09-13T23:44:29.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.057+0000 7fd313115700 1 -- 172.21.15.167:0/1567673027 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56467006e2c0 con 0x564670056c00 2024-09-13T23:44:29.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.061+0000 7fd32f992f40 1 -- 172.21.15.167:0/1567673027 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56466fbf3d40 con 0x564670056c00 2024-09-13T23:44:29.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.061+0000 7fd313115700 1 -- 172.21.15.167:0/1567673027 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56466fcdbb80 con 0x564670056c00 2024-09-13T23:44:29.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.061+0000 7fd313115700 1 --2- 172.21.15.167:0/1567673027 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5646700de000 0x5646700e4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.061+0000 7fd313115700 1 -- 172.21.15.167:0/1567673027 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x56466fc47500 con 0x564670056c00 2024-09-13T23:44:29.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.061+0000 7fd32e94c700 1 --2- 172.21.15.167:0/1567673027 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5646700de000 0x5646700e4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.061+0000 7fd32f992f40 5 note: GC not initialized 2024-09-13T23:44:29.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.061+0000 7fd32e94c700 1 --2- 172.21.15.167:0/1567673027 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5646700de000 0x5646700e4000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5646700015f0 tx=0x56466ff457a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.065+0000 7fd310910700 20 reqs_thread_entry: start 2024-09-13T23:44:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.065+0000 7fd32f992f40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.065+0000 7fd32f992f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.065+0000 7fd30e90c700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:59 2024 2024-09-13T23:44:29.070 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.065+0000 7fd32f992f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.065+0000 7fd30e0ea700 20 reqs_thread_entry: start 2024-09-13T23:44:29.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.065+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564670056000 con 0x56466ff8d000 2024-09-13T23:44:29.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.065+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.065+0000 7fd32f992f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:44:29.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.065+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564670056400 con 0x56466ff2a000 2024-09-13T23:44:29.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.065+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.071 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: "entries": [ 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: "buckets": [ 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "", 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "list_buckets", 2024-09-13T23:44:29.072 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 322, 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo", 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.073 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 5, 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 5 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 236, 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 2, 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.074 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_obj", 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 4, 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "get_acls", 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 1026, 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 2, 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 2 2024-09-13T23:44:29.075 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.076 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.076 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "get_obj", 2024-09-13T23:44:29.076 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 4, 2024-09-13T23:44:29.076 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.076 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.076 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.076 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.076 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.076 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "list_bucket", 2024-09-13T23:44:29.076 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 515, 2024-09-13T23:44:29.076 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.076 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "put_acls", 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 2, 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 2 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "put_obj", 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 30000042, 2024-09-13T23:44:29.077 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 11, 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 11 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo2", 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:29.078 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.079 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.069+0000 7fd315119700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:29.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.069+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94860527124480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2a800 con 0x56466ff8d000 2024-09-13T23:44:29.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.069+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94860527124480] v35'201 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.069+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94860525427456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff2b000 con 0x56466ff8d400 2024-09-13T23:44:29.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.073+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94860525427456] v35'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.073+0000 7fd32f992f40 20 remove_watcher() i=0 2024-09-13T23:44:29.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.073+0000 7fd32f992f40 2 removed watcher, disabling cache 2024-09-13T23:44:29.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.073+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94860525428608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56466ff8d800 con 0x56466ff8d000 2024-09-13T23:44:29.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.073+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94860525428608] v35'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff8d000 2024-09-13T23:44:29.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.073+0000 7fd32f992f40 20 remove_watcher() i=1 2024-09-13T23:44:29.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.073+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94860525429760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5646700de400 con 0x56466ff8d400 2024-09-13T23:44:29.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.077+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94860525429760] v35'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.077+0000 7fd32f992f40 20 remove_watcher() i=2 2024-09-13T23:44:29.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.077+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94860525430912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5646700de800 con 0x56466ff2a400 2024-09-13T23:44:29.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.077+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94860525430912] v35'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.077+0000 7fd32f992f40 20 remove_watcher() i=3 2024-09-13T23:44:29.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.077+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94860525432064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5646700dec00 con 0x56466ff8d400 2024-09-13T23:44:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.081+0000 7fd32e14b700 1 -- 172.21.15.167:0/3433621883 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94860525432064] v35'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5200 con 0x56466ff8d400 2024-09-13T23:44:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.081+0000 7fd32f992f40 20 remove_watcher() i=4 2024-09-13T23:44:29.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.081+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94860525433216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5646700df000 con 0x56466ff2a000 2024-09-13T23:44:29.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.081+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94860525433216] v35'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.081+0000 7fd32f992f40 20 remove_watcher() i=5 2024-09-13T23:44:29.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.081+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94860525434368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5646700df400 con 0x56466ff2a000 2024-09-13T23:44:29.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94860525434368] v35'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe5440 con 0x56466ff2a000 2024-09-13T23:44:29.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 20 remove_watcher() i=6 2024-09-13T23:44:29.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94860525435520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5646700df800 con 0x56466ff2a400 2024-09-13T23:44:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32e94c700 1 -- 172.21.15.167:0/3433621883 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94860525435520] v35'197 uv195 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56466ffe4b40 con 0x56466ff2a400 2024-09-13T23:44:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 20 remove_watcher() i=7 2024-09-13T23:44:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56466ff8d400 msgr2=0x56466fec8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56466ff8d400 0x56466fec8100 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56466ff8d000 msgr2=0x56466fec7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56466ff8d000 0x56466fec7b80 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56466ff2a400 msgr2=0x56466fec8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56466ff2a400 0x56466fec8c00 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56466ff2a000 msgr2=0x56466fec8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56466ff2a000 0x56466fec8680 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56466ff8cc00 msgr2=0x56466fec7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56466ff8cc00 0x56466fec7600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x56466fec1920 tx=0x56466fce5800 comp rx=0 tx=0).stop 2024-09-13T23:44:29.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 msgr2=0x56466fec6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 0x56466fec6b00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x56466ff10840 tx=0x56466ff44840 comp rx=0 tx=0).stop 2024-09-13T23:44:29.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.085+0000 7fd32d94a700 1 -- 172.21.15.167:0/3433621883 reap_dead start 2024-09-13T23:44:29.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 shutdown_connections 2024-09-13T23:44:29.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3433621883 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1800 0x56466fec6b00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 >> 172.21.15.167:0/3433621883 conn(0x56466fbe1400 msgr2=0x56466fce1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 shutdown_connections 2024-09-13T23:44:29.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/3433621883 wait complete. 2024-09-13T23:44:29.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/1567673027 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5646700de000 msgr2=0x5646700e4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 --2- 172.21.15.167:0/1567673027 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5646700de000 0x5646700e4000 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5646700015f0 tx=0x56466ff457a0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/1567673027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 msgr2=0x56466fec9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 --2- 172.21.15.167:0/1567673027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 0x56466fec9180 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x564670064390 tx=0x564670036420 comp rx=0 tx=0).stop 2024-09-13T23:44:29.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/1567673027 shutdown_connections 2024-09-13T23:44:29.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 --2- 172.21.15.167:0/1567673027 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5646700de000 0x5646700e4000 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 --2- 172.21.15.167:0/1567673027 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564670056c00 0x56466fec9180 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/1567673027 >> 172.21.15.167:0/1567673027 conn(0x564670057c00 msgr2=0x56466fce2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/1567673027 shutdown_connections 2024-09-13T23:44:29.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/1567673027 wait complete. 2024-09-13T23:44:29.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/3572748319 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56466ff8c000 msgr2=0x56466fec7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3572748319 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56466ff8c000 0x56466fec7080 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x56466ff10f60 tx=0x56466ff44240 comp rx=0 tx=0).stop 2024-09-13T23:44:29.096 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/3572748319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 msgr2=0x56466fec6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3572748319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 0x56466fec6580 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x56466fd3def0 tx=0x56466fce5a40 comp rx=0 tx=0).stop 2024-09-13T23:44:29.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/3572748319 shutdown_connections 2024-09-13T23:44:29.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3572748319 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56466ff8c000 0x56466fec7080 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 --2- 172.21.15.167:0/3572748319 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56466fbe1000 0x56466fec6580 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/3572748319 >> 172.21.15.167:0/3572748319 conn(0x56466fbe0c00 msgr2=0x56466fce1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/3572748319 shutdown_connections 2024-09-13T23:44:29.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.089+0000 7fd32f992f40 1 -- 172.21.15.167:0/3572748319 wait complete. 2024-09-13T23:44:29.097 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo3", 2024-09-13T23:44:29.097 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:29.098 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo4", 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.099 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.100 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: "summary": [ 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.101 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 8, 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 8 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 236, 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 5, 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_obj", 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.102 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 4, 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "get_acls", 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 1026, 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 2, 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 2 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "get_obj", 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 4, 2024-09-13T23:44:29.103 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "list_bucket", 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 515, 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "list_buckets", 2024-09-13T23:44:29.104 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 322, 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "put_acls", 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 2, 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 2 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "put_obj", 2024-09-13T23:44:29.105 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 30000042, 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 11, 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 11 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: "total": { 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 2103, 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 30000042, 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 35, 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 34 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.106 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:44:29.107 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34}}]} 2024-09-13T23:44:29.108 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-09-13T23:44:29.108 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-09-13T23:44:29.108 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories create_bucket 2024-09-13T23:44:29.153 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:29.154 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:29.154 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.149+0000 7f84ee913f40 1 Processor -- start 2024-09-13T23:44:29.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.149+0000 7f84ee913f40 1 -- start start 2024-09-13T23:44:29.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.149+0000 7f84ee913f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55bc875df400 0x55bc878c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.149+0000 7f84ee913f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55bc876a3500 con 0x55bc875df000 2024-09-13T23:44:29.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.149+0000 7f84ee913f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55bc8766ca00 con 0x55bc875df400 2024-09-13T23:44:29.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.149+0000 7f84ec8cb700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55bc875df400 0x55bc878c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.149+0000 7f84ec8cb700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55bc875df400 0x55bc878c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42522/0 (socket says 172.21.15.167:42522) 2024-09-13T23:44:29.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.149+0000 7f84ec8cb700 1 -- 172.21.15.167:0/1667777069 learned_addr learned my addr 172.21.15.167:0/1667777069 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.155 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.149+0000 7f84ed0cc700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55bc875df000 0x55bc876de800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:36388/0 (socket says 172.21.15.167:36388) 2024-09-13T23:44:29.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.149+0000 7f84ec8cb700 1 -- 172.21.15.167:0/1667777069 >> v1:172.21.15.12:6789/0 conn(0x55bc875df000 legacy=0x55bc876de800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.149+0000 7f84ec8cb700 1 -- 172.21.15.167:0/1667777069 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc875f1d40 con 0x55bc875df400 2024-09-13T23:44:29.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.149+0000 7f84ec8cb700 1 --2- 172.21.15.167:0/1667777069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df400 0x55bc878c4580 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x55bc878bf920 tx=0x55bc876e3920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b792f7554533313 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ec0ca700 1 -- 172.21.15.167:0/1667777069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc876834a0 con 0x55bc875df400 2024-09-13T23:44:29.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ec0ca700 1 -- 172.21.15.167:0/1667777069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc876a3800 con 0x55bc875df400 2024-09-13T23:44:29.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ec0ca700 1 -- 172.21.15.167:0/1667777069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc87683600 con 0x55bc875df400 2024-09-13T23:44:29.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ee913f40 1 -- 172.21.15.167:0/1667777069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df400 msgr2=0x55bc878c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ee913f40 1 --2- 172.21.15.167:0/1667777069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df400 0x55bc878c4580 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x55bc878bf920 tx=0x55bc876e3920 comp rx=0 tx=0).stop 2024-09-13T23:44:29.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ee913f40 1 -- 172.21.15.167:0/1667777069 shutdown_connections 2024-09-13T23:44:29.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ee913f40 1 --2- 172.21.15.167:0/1667777069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df400 0x55bc878c4580 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ee913f40 1 -- 172.21.15.167:0/1667777069 >> 172.21.15.167:0/1667777069 conn(0x55bc875dec00 msgr2=0x55bc876de000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ee913f40 1 -- 172.21.15.167:0/1667777069 shutdown_connections 2024-09-13T23:44:29.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ee913f40 1 -- 172.21.15.167:0/1667777069 wait complete. 2024-09-13T23:44:29.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ee913f40 1 Processor -- start 2024-09-13T23:44:29.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ee913f40 1 -- start start 2024-09-13T23:44:29.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ee913f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 0x55bc878c4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ee913f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8766c500 con 0x55bc875df000 2024-09-13T23:44:29.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ed0cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 0x55bc878c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ed0cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 0x55bc878c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42534/0 (socket says 172.21.15.167:42534) 2024-09-13T23:44:29.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ed0cc700 1 -- 172.21.15.167:0/2180082335 learned_addr learned my addr 172.21.15.167:0/2180082335 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ed0cc700 1 -- 172.21.15.167:0/2180082335 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc875f1ba0 con 0x55bc875df000 2024-09-13T23:44:29.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.153+0000 7f84ed0cc700 1 --2- 172.21.15.167:0/2180082335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 0x55bc878c4580 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x55bc8773be90 tx=0x55bc876e3b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35eaafaeadbe104 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84eb8c9700 1 -- 172.21.15.167:0/2180082335 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc876831e0 con 0x55bc875df000 2024-09-13T23:44:29.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84eb8c9700 1 -- 172.21.15.167:0/2180082335 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc876a3b00 con 0x55bc875df000 2024-09-13T23:44:29.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84eb8c9700 1 -- 172.21.15.167:0/2180082335 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc876834a0 con 0x55bc875df000 2024-09-13T23:44:29.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ee913f40 1 -- 172.21.15.167:0/2180082335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 msgr2=0x55bc878c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ee913f40 1 --2- 172.21.15.167:0/2180082335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 0x55bc878c4580 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x55bc8773be90 tx=0x55bc876e3b60 comp rx=0 tx=0).stop 2024-09-13T23:44:29.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ee913f40 1 -- 172.21.15.167:0/2180082335 shutdown_connections 2024-09-13T23:44:29.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ee913f40 1 --2- 172.21.15.167:0/2180082335 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 0x55bc878c4580 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ee913f40 1 -- 172.21.15.167:0/2180082335 >> 172.21.15.167:0/2180082335 conn(0x55bc875dec00 msgr2=0x55bc876df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ee913f40 1 -- 172.21.15.167:0/2180082335 shutdown_connections 2024-09-13T23:44:29.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ee913f40 1 -- 172.21.15.167:0/2180082335 wait complete. 2024-09-13T23:44:29.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ee913f40 1 Processor -- start 2024-09-13T23:44:29.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ee913f40 1 -- start start 2024-09-13T23:44:29.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ee913f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 0x55bc878c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ee913f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8766c3c0 con 0x55bc875df000 2024-09-13T23:44:29.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ed0cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 0x55bc878c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ed0cc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 0x55bc878c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42544/0 (socket says 172.21.15.167:42544) 2024-09-13T23:44:29.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ed0cc700 1 -- 172.21.15.167:0/1750833821 learned_addr learned my addr 172.21.15.167:0/1750833821 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.157+0000 7f84ed0cc700 1 -- 172.21.15.167:0/1750833821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc875f1a00 con 0x55bc875df000 2024-09-13T23:44:29.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ed0cc700 1 --2- 172.21.15.167:0/1750833821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 0x55bc878c4580 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x55bc8773bfb0 tx=0x55bc876e3a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ea0c6700 1 -- 172.21.15.167:0/1750833821 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc87683600 con 0x55bc875df000 2024-09-13T23:44:29.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ea0c6700 1 -- 172.21.15.167:0/1750833821 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc876a3c80 con 0x55bc875df000 2024-09-13T23:44:29.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ee913f40 1 -- 172.21.15.167:0/1750833821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bc875f1860 con 0x55bc875df000 2024-09-13T23:44:29.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ea0c6700 1 -- 172.21.15.167:0/1750833821 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc876831e0 con 0x55bc875df000 2024-09-13T23:44:29.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ee913f40 1 -- 172.21.15.167:0/1750833821 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc875f16c0 con 0x55bc875df000 2024-09-13T23:44:29.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ee913f40 1 Processor -- start 2024-09-13T23:44:29.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ee913f40 1 -- start start 2024-09-13T23:44:29.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ea0c6700 1 -- 172.21.15.167:0/1750833821 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55bc876d8b00 con 0x55bc875df000 2024-09-13T23:44:29.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ee913f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 0x55bc878c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ee913f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8766c000 con 0x55bc875df800 2024-09-13T23:44:29.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ea0c6700 1 --2- 172.21.15.167:0/1750833821 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc8798e000 0x55bc878c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ea0c6700 1 -- 172.21.15.167:0/1750833821 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55bc87645180 con 0x55bc875df000 2024-09-13T23:44:29.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ed0cc700 1 --2- 172.21.15.167:0/1750833821 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc8798e000 0x55bc878c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ed8cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 0x55bc878c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ed8cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 0x55bc878c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42546/0 (socket says 172.21.15.167:42546) 2024-09-13T23:44:29.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.161+0000 7f84ed8cd700 1 -- 172.21.15.167:0/1965925704 learned_addr learned my addr 172.21.15.167:0/1965925704 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ed0cc700 1 --2- 172.21.15.167:0/1750833821 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc8798e000 0x55bc878c5080 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55bc87970480 tx=0x55bc87948240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ed8cd700 1 -- 172.21.15.167:0/1965925704 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc875f0000 con 0x55bc875df800 2024-09-13T23:44:29.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ed8cd700 1 --2- 172.21.15.167:0/1965925704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 0x55bc878c4b00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x55bc878fe330 tx=0x55bc879490e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2932920ec95c513f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84e78c1700 1 -- 172.21.15.167:0/1965925704 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc87683600 con 0x55bc875df800 2024-09-13T23:44:29.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84e78c1700 1 -- 172.21.15.167:0/1965925704 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc879a4000 con 0x55bc875df800 2024-09-13T23:44:29.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84e78c1700 1 -- 172.21.15.167:0/1965925704 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc876831e0 con 0x55bc875df800 2024-09-13T23:44:29.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ee913f40 1 -- 172.21.15.167:0/1965925704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 msgr2=0x55bc878c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ee913f40 1 --2- 172.21.15.167:0/1965925704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 0x55bc878c4b00 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x55bc878fe330 tx=0x55bc879490e0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ee913f40 1 -- 172.21.15.167:0/1965925704 shutdown_connections 2024-09-13T23:44:29.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ee913f40 1 --2- 172.21.15.167:0/1965925704 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 0x55bc878c4b00 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ee913f40 1 -- 172.21.15.167:0/1965925704 >> 172.21.15.167:0/1965925704 conn(0x55bc875df400 msgr2=0x55bc876df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ee913f40 1 -- 172.21.15.167:0/1965925704 shutdown_connections 2024-09-13T23:44:29.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ee913f40 1 -- 172.21.15.167:0/1965925704 wait complete. 2024-09-13T23:44:29.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ee913f40 1 Processor -- start 2024-09-13T23:44:29.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ee913f40 1 -- start start 2024-09-13T23:44:29.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ee913f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 0x55bc878c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ee913f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc87604640 con 0x55bc875df800 2024-09-13T23:44:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ed8cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 0x55bc878c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ed8cd700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 0x55bc878c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42560/0 (socket says 172.21.15.167:42560) 2024-09-13T23:44:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 learned_addr learned my addr 172.21.15.167:0/3014944669 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc875f0ea0 con 0x55bc875df800 2024-09-13T23:44:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84ed8cd700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 0x55bc878c4b00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x55bc878fe960 tx=0x55bc87948cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.165+0000 7f84e60be700 1 -- 172.21.15.167:0/3014944669 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc87683a20 con 0x55bc875df800 2024-09-13T23:44:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84e60be700 1 -- 172.21.15.167:0/3014944669 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc879a4180 con 0x55bc875df800 2024-09-13T23:44:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84e60be700 1 -- 172.21.15.167:0/3014944669 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc87683600 con 0x55bc875df800 2024-09-13T23:44:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bc875f0820 con 0x55bc875df800 2024-09-13T23:44:29.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc875f09c0 con 0x55bc875df800 2024-09-13T23:44:29.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ee913f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84e60be700 1 -- 172.21.15.167:0/3014944669 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55bc876d8dc0 con 0x55bc875df800 2024-09-13T23:44:29.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84e60be700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc8798ec00 0x55bc878c5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84e60be700 1 -- 172.21.15.167:0/3014944669 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55bc87645340 con 0x55bc875df800 2024-09-13T23:44:29.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ec8cb700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc8798ec00 0x55bc878c5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ee913f40 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bc8798f000 0x55bc878c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f400 con 0x55bc8798f000 2024-09-13T23:44:29.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ed8cd700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bc8798f000 0x55bc878c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ec8cb700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc8798ec00 0x55bc878c5600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55bc878bf920 tx=0x55bc876e3800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ed8cd700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bc8798f000 0x55bc878c5b80 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ee913f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ee913f40 20 realm 2024-09-13T23:44:29.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ee913f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.169+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f000 2024-09-13T23:44:29.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:29.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798fc00 con 0x55bc8798f000 2024-09-13T23:44:29.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bc8798f400 0x55bc878c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f400 2024-09-13T23:44:29.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ed0cc700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bc8798f400 0x55bc878c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ed0cc700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bc8798f400 0x55bc878c6100 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798fc00 con 0x55bc8798f400 2024-09-13T23:44:29.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:29.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 20 searching for the correct realm 2024-09-13T23:44:29.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bc87938000 0x55bc878c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87938400 con 0x55bc87938000 2024-09-13T23:44:29.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.173+0000 7f84ec8cb700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bc87938000 0x55bc878c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.177+0000 7f84ec8cb700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bc87938000 0x55bc878c6680 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.177+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.177+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87938400 con 0x55bc8798f400 2024-09-13T23:44:29.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.177+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.177+0000 7f84ed0cc700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bc87938400 0x55bc878c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.177+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938400 2024-09-13T23:44:29.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.177+0000 7f84ed8cd700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bc87938400 0x55bc878c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.177+0000 7f84ed8cd700 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bc87938400 0x55bc878c6c00 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.177+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.177+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f000 2024-09-13T23:44:29.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.177+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.177+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f000 2024-09-13T23:44:29.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938000 2024-09-13T23:44:29.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f000 2024-09-13T23:44:29.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f400 2024-09-13T23:44:29.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f400 2024-09-13T23:44:29.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f000 2024-09-13T23:44:29.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938400 2024-09-13T23:44:29.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.181+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f000 2024-09-13T23:44:29.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.185+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.185+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938400 2024-09-13T23:44:29.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.185+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.185+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f400 2024-09-13T23:44:29.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.185+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.185+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938000 2024-09-13T23:44:29.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.185+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.185+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f000 2024-09-13T23:44:29.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.185+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.185+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f400 2024-09-13T23:44:29.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.185+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.185+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938000 2024-09-13T23:44:29.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f000 2024-09-13T23:44:29.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938400 2024-09-13T23:44:29.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f000 2024-09-13T23:44:29.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938000 2024-09-13T23:44:29.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938400 2024-09-13T23:44:29.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f000 2024-09-13T23:44:29.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.189+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938400 2024-09-13T23:44:29.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.193+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.193+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938000 2024-09-13T23:44:29.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.193+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.193+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938400 2024-09-13T23:44:29.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.193+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.193+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f400 2024-09-13T23:44:29.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.193+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.193+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f400 2024-09-13T23:44:29.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.193+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.193+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938000 2024-09-13T23:44:29.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.193+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.193+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f400 2024-09-13T23:44:29.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f400 2024-09-13T23:44:29.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:29.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:29.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:29.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:29.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87938800 con 0x55bc8798f000 2024-09-13T23:44:29.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87938c00 con 0x55bc8798f400 2024-09-13T23:44:29.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:29.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87939000 con 0x55bc8798f400 2024-09-13T23:44:29.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 zone default found 2024-09-13T23:44:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 4 Realm: () 2024-09-13T23:44:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 zonegroup default 2024-09-13T23:44:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.197+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87939400 con 0x55bc87938400 2024-09-13T23:44:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ee913f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ee913f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87939800 con 0x55bc87938000 2024-09-13T23:44:29.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ee913f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ee913f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87939c00 con 0x55bc87938000 2024-09-13T23:44:29.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ee913f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ee913f40 20 started sync module instance, tier type = 2024-09-13T23:44:29.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ee913f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:29.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.201+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87939800 con 0x55bc8798f400 2024-09-13T23:44:29.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.205+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.205+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94268216294144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87939c00 con 0x55bc8798f400 2024-09-13T23:44:29.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.205+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87938c00 con 0x55bc8798f000 2024-09-13T23:44:29.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.205+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94268216294144] v35'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.205+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.205+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94268216295296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87939000 con 0x55bc8798f000 2024-09-13T23:44:29.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.205+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87939800 con 0x55bc8798f400 2024-09-13T23:44:29.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.209+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94268216295296] v35'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.209+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.209+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94268216296448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798fc00 con 0x55bc8798f400 2024-09-13T23:44:29.215 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.209+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc87938400 2024-09-13T23:44:29.217 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.213+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.213+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94268216297600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87938800 con 0x55bc87938400 2024-09-13T23:44:29.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.213+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87939400 con 0x55bc8798f400 2024-09-13T23:44:29.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.213+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94268216296448] v35'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.219 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.213+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94268216297600] v35'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.213+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.213+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94268216298752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87938c00 con 0x55bc8798f400 2024-09-13T23:44:29.220 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.213+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87939000 con 0x55bc87938000 2024-09-13T23:44:29.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.217+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.217+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94268216298752] v35'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.217+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94268216299904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87a5e000 con 0x55bc87938000 2024-09-13T23:44:29.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.217+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87a5e400 con 0x55bc87938000 2024-09-13T23:44:29.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94268216299904] v35'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94268216301056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87a5e800 con 0x55bc87938000 2024-09-13T23:44:29.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87a5ec00 con 0x55bc87938400 2024-09-13T23:44:29.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94268216301056] v35'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'198 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94268216302208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87a5f000 con 0x55bc87938400 2024-09-13T23:44:29.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ee913f40 20 add_watcher() i=0 2024-09-13T23:44:29.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ee913f40 20 add_watcher() i=1 2024-09-13T23:44:29.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ee913f40 20 add_watcher() i=2 2024-09-13T23:44:29.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ee913f40 20 add_watcher() i=3 2024-09-13T23:44:29.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ee913f40 20 add_watcher() i=4 2024-09-13T23:44:29.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ee913f40 20 add_watcher() i=5 2024-09-13T23:44:29.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.221+0000 7f84ee913f40 20 add_watcher() i=6 2024-09-13T23:44:29.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.225+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94268216302208] v35'199 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.225+0000 7f84ee913f40 20 add_watcher() i=7 2024-09-13T23:44:29.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.225+0000 7f84ee913f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:29.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.225+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87a5f400 con 0x55bc8798f000 2024-09-13T23:44:29.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.225+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.225+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94268218032128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87a5f800 con 0x55bc8798f000 2024-09-13T23:44:29.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.225+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94268218032128] v35'202 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.225+0000 7f84d409a700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:29.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.225+0000 7f84ee913f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:29.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.225+0000 7f84ee913f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:29.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ee913f40 1 Processor -- start 2024-09-13T23:44:29.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ee913f40 1 -- start start 2024-09-13T23:44:29.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ee913f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 0x55bc878c7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ee913f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc87604780 con 0x55bc87a5ec00 2024-09-13T23:44:29.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ec8cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 0x55bc878c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ec8cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 0x55bc878c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42568/0 (socket says 172.21.15.167:42568) 2024-09-13T23:44:29.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3143678951 learned_addr learned my addr 172.21.15.167:0/3143678951 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3143678951 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc875f0b60 con 0x55bc87a5ec00 2024-09-13T23:44:29.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ec8cb700 1 --2- 172.21.15.167:0/3143678951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 0x55bc878c7180 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55bc878fe1b0 tx=0x55bc876e3860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d4589f26beb3b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84d3899700 1 -- 172.21.15.167:0/3143678951 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc87683760 con 0x55bc87a5ec00 2024-09-13T23:44:29.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84d3899700 1 -- 172.21.15.167:0/3143678951 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc879a4600 con 0x55bc87a5ec00 2024-09-13T23:44:29.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84d3899700 1 -- 172.21.15.167:0/3143678951 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc87a70000 con 0x55bc87a5ec00 2024-09-13T23:44:29.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ee913f40 1 -- 172.21.15.167:0/3143678951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 msgr2=0x55bc878c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ee913f40 1 --2- 172.21.15.167:0/3143678951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 0x55bc878c7180 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x55bc878fe1b0 tx=0x55bc876e3860 comp rx=0 tx=0).stop 2024-09-13T23:44:29.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ee913f40 1 -- 172.21.15.167:0/3143678951 shutdown_connections 2024-09-13T23:44:29.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ee913f40 1 --2- 172.21.15.167:0/3143678951 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 0x55bc878c7180 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ee913f40 1 -- 172.21.15.167:0/3143678951 >> 172.21.15.167:0/3143678951 conn(0x55bc87a5fc00 msgr2=0x55bc876e0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ee913f40 1 -- 172.21.15.167:0/3143678951 shutdown_connections 2024-09-13T23:44:29.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.229+0000 7f84ee913f40 1 -- 172.21.15.167:0/3143678951 wait complete. 2024-09-13T23:44:29.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ee913f40 1 Processor -- start 2024-09-13T23:44:29.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ee913f40 1 -- start start 2024-09-13T23:44:29.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ee913f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 0x55bc878c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ee913f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc8766c000 con 0x55bc87a5ec00 2024-09-13T23:44:29.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ec8cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 0x55bc878c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ec8cb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 0x55bc878c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42576/0 (socket says 172.21.15.167:42576) 2024-09-13T23:44:29.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ec8cb700 1 -- 172.21.15.167:0/770649886 learned_addr learned my addr 172.21.15.167:0/770649886 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ec8cb700 1 -- 172.21.15.167:0/770649886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc875f0d00 con 0x55bc87a5ec00 2024-09-13T23:44:29.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ec8cb700 1 --2- 172.21.15.167:0/770649886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 0x55bc878c7180 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x55bc87a6e210 tx=0x55bc87a3e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84d2096700 1 -- 172.21.15.167:0/770649886 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc87a70160 con 0x55bc87a5ec00 2024-09-13T23:44:29.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ee913f40 1 -- 172.21.15.167:0/770649886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bc875f0340 con 0x55bc87a5ec00 2024-09-13T23:44:29.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84d2096700 1 -- 172.21.15.167:0/770649886 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bc879a4300 con 0x55bc87a5ec00 2024-09-13T23:44:29.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84d2096700 1 -- 172.21.15.167:0/770649886 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55bc87a702c0 con 0x55bc87a5ec00 2024-09-13T23:44:29.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ee913f40 1 -- 172.21.15.167:0/770649886 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc875f1d40 con 0x55bc87a5ec00 2024-09-13T23:44:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84d2096700 1 -- 172.21.15.167:0/770649886 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55bc876d9b80 con 0x55bc87a5ec00 2024-09-13T23:44:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84d2096700 1 --2- 172.21.15.167:0/770649886 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc87ae8000 0x55bc87aee000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84d2096700 1 -- 172.21.15.167:0/770649886 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55bc87645500 con 0x55bc87a5ec00 2024-09-13T23:44:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ed8cd700 1 --2- 172.21.15.167:0/770649886 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc87ae8000 0x55bc87aee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.233+0000 7f84ee913f40 5 note: GC not initialized 2024-09-13T23:44:29.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.237+0000 7f84ed8cd700 1 --2- 172.21.15.167:0/770649886 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc87ae8000 0x55bc87aee000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55bc879efbc0 tx=0x55bc879497a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.237+0000 7f84cf891700 20 reqs_thread_entry: start 2024-09-13T23:44:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.237+0000 7f84ee913f40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.237+0000 7f84ee913f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.237+0000 7f84cd88d700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:59 2024 2024-09-13T23:44:29.243 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.237+0000 7f84cd06b700 20 reqs_thread_entry: start 2024-09-13T23:44:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.237+0000 7f84ee913f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.237+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87a5e000 con 0x55bc8798f000 2024-09-13T23:44:29.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.237+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.237+0000 7f84ee913f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:44:29.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.237+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87a5e400 con 0x55bc87938000 2024-09-13T23:44:29.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.237+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.241+0000 7f84d409a700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:29.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.241+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94268218032128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87938800 con 0x55bc8798f000 2024-09-13T23:44:29.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.241+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94268218032128] v35'203 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.241+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94268216294144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87939000 con 0x55bc8798f400 2024-09-13T23:44:29.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.245+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94268216294144] v35'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.245+0000 7f84ee913f40 20 remove_watcher() i=0 2024-09-13T23:44:29.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.245+0000 7f84ee913f40 2 removed watcher, disabling cache 2024-09-13T23:44:29.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.245+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94268216295296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc8798f800 con 0x55bc8798f000 2024-09-13T23:44:29.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.249+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94268216295296] v35'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc8798f000 2024-09-13T23:44:29.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.249+0000 7f84ee913f40 20 remove_watcher() i=1 2024-09-13T23:44:29.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.249+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94268216296448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87ae8400 con 0x55bc8798f400 2024-09-13T23:44:29.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.249+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94268216296448] v35'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.249+0000 7f84ee913f40 20 remove_watcher() i=2 2024-09-13T23:44:29.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.249+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94268216297600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87ae8800 con 0x55bc87938400 2024-09-13T23:44:29.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.253+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94268216297600] v35'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.253+0000 7f84ee913f40 20 remove_watcher() i=3 2024-09-13T23:44:29.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.253+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94268216298752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87ae8c00 con 0x55bc8798f400 2024-09-13T23:44:29.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.257+0000 7f84ed0cc700 1 -- 172.21.15.167:0/3014944669 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94268216298752] v35'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d200 con 0x55bc8798f400 2024-09-13T23:44:29.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.257+0000 7f84ee913f40 20 remove_watcher() i=4 2024-09-13T23:44:29.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.257+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94268216299904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87ae9000 con 0x55bc87938000 2024-09-13T23:44:29.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.257+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94268216299904] v35'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.257+0000 7f84ee913f40 20 remove_watcher() i=5 2024-09-13T23:44:29.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.257+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94268216301056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87ae9400 con 0x55bc87938000 2024-09-13T23:44:29.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94268216301056] v35'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2d440 con 0x55bc87938000 2024-09-13T23:44:29.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ee913f40 20 remove_watcher() i=6 2024-09-13T23:44:29.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94268216302208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55bc87ae9800 con 0x55bc87938400 2024-09-13T23:44:29.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ed8cd700 1 -- 172.21.15.167:0/3014944669 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94268216302208] v35'200 uv198 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55bc87a2cb40 con 0x55bc87938400 2024-09-13T23:44:29.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ee913f40 20 remove_watcher() i=7 2024-09-13T23:44:29.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bc8798f400 msgr2=0x55bc878c6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ee913f40 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55bc8798f400 0x55bc878c6100 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bc8798f000 msgr2=0x55bc878c5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ee913f40 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55bc8798f000 0x55bc878c5b80 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bc87938400 msgr2=0x55bc878c6c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ee913f40 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55bc87938400 0x55bc878c6c00 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bc87938000 msgr2=0x55bc878c6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.261+0000 7f84ee913f40 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55bc87938000 0x55bc878c6680 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc8798ec00 msgr2=0x55bc878c5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc8798ec00 0x55bc878c5600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x55bc878bf920 tx=0x55bc876e3800 comp rx=0 tx=0).stop 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 msgr2=0x55bc878c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 --2- 172.21.15.167:0/3014944669 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df800 0x55bc878c4b00 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x55bc878fe960 tx=0x55bc87948cc0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 reap_dead start 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ec8cb700 1 -- 172.21.15.167:0/3014944669 reap_dead start 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 shutdown_connections 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 >> 172.21.15.167:0/3014944669 conn(0x55bc875df400 msgr2=0x55bc876df800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 shutdown_connections 2024-09-13T23:44:29.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/3014944669 wait complete. 2024-09-13T23:44:29.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/770649886 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc87ae8000 msgr2=0x55bc87aee000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 --2- 172.21.15.167:0/770649886 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc87ae8000 0x55bc87aee000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55bc879efbc0 tx=0x55bc879497a0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/770649886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 msgr2=0x55bc878c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 --2- 172.21.15.167:0/770649886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 0x55bc878c7180 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x55bc87a6e210 tx=0x55bc87a3e420 comp rx=0 tx=0).stop 2024-09-13T23:44:29.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/770649886 shutdown_connections 2024-09-13T23:44:29.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 --2- 172.21.15.167:0/770649886 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc87ae8000 0x55bc87aee000 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 --2- 172.21.15.167:0/770649886 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc87a5ec00 0x55bc878c7180 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/770649886 >> 172.21.15.167:0/770649886 conn(0x55bc87a5fc00 msgr2=0x55bc876e0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/770649886 shutdown_connections 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/770649886 wait complete. 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/1750833821 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc8798e000 msgr2=0x55bc878c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 --2- 172.21.15.167:0/1750833821 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc8798e000 0x55bc878c5080 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55bc87970480 tx=0x55bc87948240 comp rx=0 tx=0).stop 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/1750833821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 msgr2=0x55bc878c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 --2- 172.21.15.167:0/1750833821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 0x55bc878c4580 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x55bc8773bfb0 tx=0x55bc876e3a40 comp rx=0 tx=0).stop 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/1750833821 shutdown_connections 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 --2- 172.21.15.167:0/1750833821 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55bc8798e000 0x55bc878c5080 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 --2- 172.21.15.167:0/1750833821 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55bc875df000 0x55bc878c4580 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/1750833821 >> 172.21.15.167:0/1750833821 conn(0x55bc875dec00 msgr2=0x55bc876df000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/1750833821 shutdown_connections 2024-09-13T23:44:29.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.265+0000 7f84ee913f40 1 -- 172.21.15.167:0/1750833821 wait complete. 2024-09-13T23:44:29.274 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:44:29.274 INFO:teuthology.orchestra.run.smithi167.stdout: "entries": [ 2024-09-13T23:44:29.274 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.274 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:29.274 INFO:teuthology.orchestra.run.smithi167.stdout: "buckets": [ 2024-09-13T23:44:29.274 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.274 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "", 2024-09-13T23:44:29.274 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.274 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo", 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.275 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 5, 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 5 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo2", 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:29.276 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo3", 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.277 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo4", 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.278 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: "summary": [ 2024-09-13T23:44:29.279 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "create_bucket", 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 8, 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 8 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: "total": { 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.280 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 8, 2024-09-13T23:44:29.281 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 8 2024-09-13T23:44:29.281 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.281 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.281 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.281 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:44:29.282 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}]}, {'bucket': 'myfoo2', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}}]} 2024-09-13T23:44:29.282 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-09-13T23:44:29.282 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-09-13T23:44:29.282 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories put_obj 2024-09-13T23:44:29.366 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:29.366 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:29.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.361+0000 7fcbf364df40 1 Processor -- start 2024-09-13T23:44:29.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.361+0000 7fcbf364df40 1 -- start start 2024-09-13T23:44:29.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.361+0000 7fcbf364df40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x563a5e2f7000 0x563a5e5dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.361+0000 7fcbf364df40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x563a5e3bb500 con 0x563a5e2f7400 2024-09-13T23:44:29.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.361+0000 7fcbf364df40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x563a5e384a00 con 0x563a5e2f7000 2024-09-13T23:44:29.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.361+0000 7fcbf1e06700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x563a5e2f7000 0x563a5e5dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.361+0000 7fcbf1605700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x563a5e2f7400 0x563a5e3f6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:36390/0 (socket says 172.21.15.167:36390) 2024-09-13T23:44:29.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.361+0000 7fcbf1605700 1 -- 172.21.15.167:0/4124725702 learned_addr learned my addr 172.21.15.167:0/4124725702 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:44:29.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.361+0000 7fcbf1e06700 1 -- 172.21.15.167:0/4124725702 >> v1:172.21.15.12:6789/0 conn(0x563a5e2f7400 legacy=0x563a5e3f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.361+0000 7fcbf0e04700 1 -- 172.21.15.167:0/4124725702 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647863352 0 0) 0x563a5e452000 con 0x563a5e2f7400 2024-09-13T23:44:29.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.361+0000 7fcbf1e06700 1 -- 172.21.15.167:0/4124725702 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a5e309d40 con 0x563a5e2f7000 2024-09-13T23:44:29.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf1e06700 1 --2- 172.21.15.167:0/4124725702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7000 0x563a5e5dc580 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x563a5e5d7a70 tx=0x563a5e3fb920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190699af1c8d968e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf0e04700 1 -- 172.21.15.167:0/4124725702 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e39b4a0 con 0x563a5e2f7000 2024-09-13T23:44:29.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf0e04700 1 -- 172.21.15.167:0/4124725702 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a5e3bb800 con 0x563a5e2f7000 2024-09-13T23:44:29.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf0e04700 1 -- 172.21.15.167:0/4124725702 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e39b600 con 0x563a5e2f7000 2024-09-13T23:44:29.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf364df40 1 -- 172.21.15.167:0/4124725702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7000 msgr2=0x563a5e5dc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf364df40 1 --2- 172.21.15.167:0/4124725702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7000 0x563a5e5dc580 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x563a5e5d7a70 tx=0x563a5e3fb920 comp rx=0 tx=0).stop 2024-09-13T23:44:29.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf364df40 1 -- 172.21.15.167:0/4124725702 shutdown_connections 2024-09-13T23:44:29.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf364df40 1 --2- 172.21.15.167:0/4124725702 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7000 0x563a5e5dc580 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf364df40 1 -- 172.21.15.167:0/4124725702 >> 172.21.15.167:0/4124725702 conn(0x563a5e2f6c00 msgr2=0x563a5e3f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf364df40 1 -- 172.21.15.167:0/4124725702 shutdown_connections 2024-09-13T23:44:29.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf364df40 1 -- 172.21.15.167:0/4124725702 wait complete. 2024-09-13T23:44:29.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf364df40 1 Processor -- start 2024-09-13T23:44:29.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf364df40 1 -- start start 2024-09-13T23:44:29.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf364df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7000 0x563a5e5dc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf364df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5e384500 con 0x563a5e2f7000 2024-09-13T23:44:29.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf1e06700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7000 0x563a5e5dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf1e06700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7000 0x563a5e5dc580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42596/0 (socket says 172.21.15.167:42596) 2024-09-13T23:44:29.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf1e06700 1 -- 172.21.15.167:0/743033202 learned_addr learned my addr 172.21.15.167:0/743033202 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.365+0000 7fcbf1e06700 1 -- 172.21.15.167:0/743033202 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a5e309ba0 con 0x563a5e2f7000 2024-09-13T23:44:29.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf1e06700 1 --2- 172.21.15.167:0/743033202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7000 0x563a5e5dc580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x563a5f0c1e00 tx=0x563a5e3fb800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba2ba2fa2527b01 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf0603700 1 -- 172.21.15.167:0/743033202 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e39b760 con 0x563a5e2f7000 2024-09-13T23:44:29.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf0603700 1 -- 172.21.15.167:0/743033202 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a5e3bbb00 con 0x563a5e2f7000 2024-09-13T23:44:29.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf0603700 1 -- 172.21.15.167:0/743033202 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e39b4a0 con 0x563a5e2f7000 2024-09-13T23:44:29.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 -- 172.21.15.167:0/743033202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7000 msgr2=0x563a5e5dc580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 --2- 172.21.15.167:0/743033202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7000 0x563a5e5dc580 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x563a5f0c1e00 tx=0x563a5e3fb800 comp rx=0 tx=0).stop 2024-09-13T23:44:29.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 -- 172.21.15.167:0/743033202 shutdown_connections 2024-09-13T23:44:29.374 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 --2- 172.21.15.167:0/743033202 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7000 0x563a5e5dc580 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 -- 172.21.15.167:0/743033202 >> 172.21.15.167:0/743033202 conn(0x563a5e2f6c00 msgr2=0x563a5e3f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 -- 172.21.15.167:0/743033202 shutdown_connections 2024-09-13T23:44:29.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 -- 172.21.15.167:0/743033202 wait complete. 2024-09-13T23:44:29.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 Processor -- start 2024-09-13T23:44:29.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 -- start start 2024-09-13T23:44:29.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7400 0x563a5e5dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.375 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5e3843c0 con 0x563a5e2f7400 2024-09-13T23:44:29.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf1e06700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7400 0x563a5e5dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf1e06700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7400 0x563a5e5dcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42612/0 (socket says 172.21.15.167:42612) 2024-09-13T23:44:29.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf1e06700 1 -- 172.21.15.167:0/3574996295 learned_addr learned my addr 172.21.15.167:0/3574996295 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf1e06700 1 -- 172.21.15.167:0/3574996295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a5e309a00 con 0x563a5e2f7400 2024-09-13T23:44:29.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf1e06700 1 --2- 172.21.15.167:0/3574996295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7400 0x563a5e5dcb00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x563a5f0c1e90 tx=0x563a5e3fb920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbeee00700 1 -- 172.21.15.167:0/3574996295 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e39b600 con 0x563a5e2f7400 2024-09-13T23:44:29.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 -- 172.21.15.167:0/3574996295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563a5e309860 con 0x563a5e2f7400 2024-09-13T23:44:29.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbeee00700 1 -- 172.21.15.167:0/3574996295 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a5e3bbc80 con 0x563a5e2f7400 2024-09-13T23:44:29.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbeee00700 1 -- 172.21.15.167:0/3574996295 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e39b8c0 con 0x563a5e2f7400 2024-09-13T23:44:29.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.369+0000 7fcbf364df40 1 -- 172.21.15.167:0/3574996295 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a5e3096c0 con 0x563a5e2f7400 2024-09-13T23:44:29.377 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbf364df40 1 Processor -- start 2024-09-13T23:44:29.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbf364df40 1 -- start start 2024-09-13T23:44:29.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbf364df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 0x563a5e5dd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbf364df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5e384000 con 0x563a5e2f7c00 2024-09-13T23:44:29.378 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbeee00700 1 -- 172.21.15.167:0/3574996295 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x563a5e3f0b00 con 0x563a5e2f7400 2024-09-13T23:44:29.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbf2607700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 0x563a5e5dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbf2607700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 0x563a5e5dd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42628/0 (socket says 172.21.15.167:42628) 2024-09-13T23:44:29.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbf2607700 1 -- 172.21.15.167:0/86517395 learned_addr learned my addr 172.21.15.167:0/86517395 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbeee00700 1 --2- 172.21.15.167:0/3574996295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e6a2000 0x563a5e5de100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbf2607700 1 -- 172.21.15.167:0/86517395 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a5e308000 con 0x563a5e2f7c00 2024-09-13T23:44:29.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbeee00700 1 -- 172.21.15.167:0/3574996295 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x563a5e35d180 con 0x563a5e2f7400 2024-09-13T23:44:29.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbf1e06700 1 --2- 172.21.15.167:0/3574996295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e6a2000 0x563a5e5de100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbf2607700 1 --2- 172.21.15.167:0/86517395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 0x563a5e5dd080 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x563a5e67a060 tx=0x563a5e3fbce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1831bdf272c92db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbec5fb700 1 -- 172.21.15.167:0/86517395 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e39b600 con 0x563a5e2f7c00 2024-09-13T23:44:29.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbec5fb700 1 -- 172.21.15.167:0/86517395 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a5e692000 con 0x563a5e2f7c00 2024-09-13T23:44:29.380 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbec5fb700 1 -- 172.21.15.167:0/86517395 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e39b8c0 con 0x563a5e2f7c00 2024-09-13T23:44:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbf364df40 1 -- 172.21.15.167:0/86517395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 msgr2=0x563a5e5dd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf364df40 1 --2- 172.21.15.167:0/86517395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 0x563a5e5dd080 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x563a5e67a060 tx=0x563a5e3fbce0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.373+0000 7fcbf1e06700 1 --2- 172.21.15.167:0/3574996295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e6a2000 0x563a5e5de100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x563a5e61f650 tx=0x563a5e660c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf364df40 1 -- 172.21.15.167:0/86517395 shutdown_connections 2024-09-13T23:44:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf364df40 1 --2- 172.21.15.167:0/86517395 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 0x563a5e5dd080 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf364df40 1 -- 172.21.15.167:0/86517395 >> 172.21.15.167:0/86517395 conn(0x563a5e2f7800 msgr2=0x563a5e3f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.381 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf364df40 1 -- 172.21.15.167:0/86517395 shutdown_connections 2024-09-13T23:44:29.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf364df40 1 -- 172.21.15.167:0/86517395 wait complete. 2024-09-13T23:44:29.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf364df40 1 Processor -- start 2024-09-13T23:44:29.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf364df40 1 -- start start 2024-09-13T23:44:29.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf364df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 0x563a5e5dd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf364df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5e31c640 con 0x563a5e2f7c00 2024-09-13T23:44:29.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf2607700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 0x563a5e5dd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf2607700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 0x563a5e5dd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42636/0 (socket says 172.21.15.167:42636) 2024-09-13T23:44:29.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 learned_addr learned my addr 172.21.15.167:0/2891422129 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a5e308ea0 con 0x563a5e2f7c00 2024-09-13T23:44:29.383 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf2607700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 0x563a5e5dd080 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x563a5f0c1e60 tx=0x563a5e3fbbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbeadf8700 1 -- 172.21.15.167:0/2891422129 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e39b760 con 0x563a5e2f7c00 2024-09-13T23:44:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbeadf8700 1 -- 172.21.15.167:0/2891422129 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a5e692180 con 0x563a5e2f7c00 2024-09-13T23:44:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbeadf8700 1 -- 172.21.15.167:0/2891422129 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e39b600 con 0x563a5e2f7c00 2024-09-13T23:44:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563a5e308820 con 0x563a5e2f7c00 2024-09-13T23:44:29.384 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.377+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a5e3089c0 con 0x563a5e2f7c00 2024-09-13T23:44:29.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf364df40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbeadf8700 1 -- 172.21.15.167:0/2891422129 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x563a5e3f1080 con 0x563a5e2f7c00 2024-09-13T23:44:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbeadf8700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e6a2c00 0x563a5e5de680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbeadf8700 1 -- 172.21.15.167:0/2891422129 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x563a5e35d340 con 0x563a5e2f7c00 2024-09-13T23:44:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf364df40 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563a5e6a3000 0x563a5e5dd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf1605700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e6a2c00 0x563a5e5de680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.386 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e6a3400 con 0x563a5e6a3000 2024-09-13T23:44:29.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf2607700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563a5e6a3000 0x563a5e5dd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf1605700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e6a2c00 0x563a5e5de680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x563a5f0c0030 tx=0x563a5e74e0c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.387 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf2607700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563a5e6a3000 0x563a5e5dd600 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf364df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf364df40 20 realm 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf364df40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e6a3800 con 0x563a5e6a3000 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf364df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf364df40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf364df40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.381+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e6a3c00 con 0x563a5e6a3000 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf364df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf364df40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf364df40 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x563a5e6a3400 0x563a5e5ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.389 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e6a3800 con 0x563a5e6a3400 2024-09-13T23:44:29.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf1e06700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x563a5e6a3400 0x563a5e5ddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf1e06700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x563a5e6a3400 0x563a5e5ddb80 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf364df40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:29.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf364df40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.391 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e6a3c00 con 0x563a5e6a3400 2024-09-13T23:44:29.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf364df40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:29.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf364df40 20 searching for the correct realm 2024-09-13T23:44:29.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf364df40 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x563a5e6a3800 0x563a5e5dec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e6a3c00 con 0x563a5e6a3800 2024-09-13T23:44:29.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.385+0000 7fcbf1605700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x563a5e6a3800 0x563a5e5dec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.389+0000 7fcbf1605700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x563a5e6a3800 0x563a5e5dec00 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.389+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.389+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e6a3c00 con 0x563a5e6a3400 2024-09-13T23:44:29.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.389+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.389+0000 7fcbf1e06700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x563a5e6a3c00 0x563a5e5dc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.389+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3c00 2024-09-13T23:44:29.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.389+0000 7fcbf2607700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x563a5e6a3c00 0x563a5e5dc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.389+0000 7fcbf2607700 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x563a5e6a3c00 0x563a5e5dc580 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.389+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.389+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3000 2024-09-13T23:44:29.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.389+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.389+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3000 2024-09-13T23:44:29.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.393+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.393+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3800 2024-09-13T23:44:29.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.393+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.393+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3000 2024-09-13T23:44:29.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.393+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.393+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3400 2024-09-13T23:44:29.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.393+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.393+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3400 2024-09-13T23:44:29.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.393+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.393+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3000 2024-09-13T23:44:29.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.393+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.393+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3c00 2024-09-13T23:44:29.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.397+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.397+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3000 2024-09-13T23:44:29.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.397+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.397+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3c00 2024-09-13T23:44:29.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.397+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.397+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3400 2024-09-13T23:44:29.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.397+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.397+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3800 2024-09-13T23:44:29.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.397+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.397+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3000 2024-09-13T23:44:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.397+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.397+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3400 2024-09-13T23:44:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3800 2024-09-13T23:44:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3000 2024-09-13T23:44:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3c00 2024-09-13T23:44:29.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3000 2024-09-13T23:44:29.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3800 2024-09-13T23:44:29.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3c00 2024-09-13T23:44:29.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.401+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3000 2024-09-13T23:44:29.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.405+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.405+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3c00 2024-09-13T23:44:29.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.405+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.405+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3800 2024-09-13T23:44:29.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.405+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.405+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3c00 2024-09-13T23:44:29.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.405+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.405+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3400 2024-09-13T23:44:29.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.405+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.405+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3400 2024-09-13T23:44:29.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.405+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.405+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3800 2024-09-13T23:44:29.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3400 2024-09-13T23:44:29.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3400 2024-09-13T23:44:29.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf364df40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf364df40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf364df40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf364df40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf364df40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762800 con 0x563a5e6a3000 2024-09-13T23:44:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf364df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf364df40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762c00 con 0x563a5e6a3400 2024-09-13T23:44:29.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf364df40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:29.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf364df40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.409+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763000 con 0x563a5e6a3400 2024-09-13T23:44:29.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:29.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 20 zone default found 2024-09-13T23:44:29.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 4 Realm: () 2024-09-13T23:44:29.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:29.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:29.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:29.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 20 zonegroup default 2024-09-13T23:44:29.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763400 con 0x563a5e6a3c00 2024-09-13T23:44:29.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763800 con 0x563a5e6a3800 2024-09-13T23:44:29.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763c00 con 0x563a5e6a3800 2024-09-13T23:44:29.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 20 started sync module instance, tier type = 2024-09-13T23:44:29.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:29.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.413+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763800 con 0x563a5e6a3400 2024-09-13T23:44:29.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.417+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.417+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94808691260160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763c00 con 0x563a5e6a3400 2024-09-13T23:44:29.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.417+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762400 con 0x563a5e6a3000 2024-09-13T23:44:29.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.421+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.421+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94808691261312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3000 2024-09-13T23:44:29.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.421+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94808691260160] v35'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.421+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762c00 con 0x563a5e6a3400 2024-09-13T23:44:29.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.421+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94808691261312] v35'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.421+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.421+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94808691262464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763000 con 0x563a5e6a3400 2024-09-13T23:44:29.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.421+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762800 con 0x563a5e6a3c00 2024-09-13T23:44:29.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.425+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.425+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94808691263616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763400 con 0x563a5e6a3c00 2024-09-13T23:44:29.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.425+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762400 con 0x563a5e6a3400 2024-09-13T23:44:29.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.425+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94808691262464] v35'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.429+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94808691263616] v35'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.429+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.429+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94808691264768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762000 con 0x563a5e6a3400 2024-09-13T23:44:29.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.429+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763800 con 0x563a5e6a3800 2024-09-13T23:44:29.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.429+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.433+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94808691265920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763c00 con 0x563a5e6a3800 2024-09-13T23:44:29.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.433+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762c00 con 0x563a5e6a3800 2024-09-13T23:44:29.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.433+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94808691264768] v35'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.433+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94808691265920] v35'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.433+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.433+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94808691267072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763000 con 0x563a5e6a3800 2024-09-13T23:44:29.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.433+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762400 con 0x563a5e6a3c00 2024-09-13T23:44:29.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'201 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94808691268224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763800 con 0x563a5e6a3c00 2024-09-13T23:44:29.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf364df40 20 add_watcher() i=0 2024-09-13T23:44:29.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf364df40 20 add_watcher() i=1 2024-09-13T23:44:29.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf364df40 20 add_watcher() i=2 2024-09-13T23:44:29.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf364df40 20 add_watcher() i=3 2024-09-13T23:44:29.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf364df40 20 add_watcher() i=4 2024-09-13T23:44:29.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf364df40 20 add_watcher() i=5 2024-09-13T23:44:29.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94808691267072] v35'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf364df40 20 add_watcher() i=6 2024-09-13T23:44:29.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94808691268224] v35'202 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf364df40 20 add_watcher() i=7 2024-09-13T23:44:29.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf364df40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:29.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e763c00 con 0x563a5e6a3000 2024-09-13T23:44:29.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.437+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94808692932608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e762c00 con 0x563a5e6a3000 2024-09-13T23:44:29.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94808692932608] v35'204 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbd8dd4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:29.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbf364df40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:29.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbf364df40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:29.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbf364df40 1 Processor -- start 2024-09-13T23:44:29.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbf364df40 1 -- start start 2024-09-13T23:44:29.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbf364df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e763400 0x563a5e5df180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbf364df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5e31c780 con 0x563a5e763400 2024-09-13T23:44:29.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbf1605700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e763400 0x563a5e5df180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbf1605700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e763400 0x563a5e5df180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42652/0 (socket says 172.21.15.167:42652) 2024-09-13T23:44:29.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbf1605700 1 -- 172.21.15.167:0/2829589012 learned_addr learned my addr 172.21.15.167:0/2829589012 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbf1605700 1 -- 172.21.15.167:0/2829589012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a5e308b60 con 0x563a5e763400 2024-09-13T23:44:29.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbf1605700 1 --2- 172.21.15.167:0/2829589012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e763400 0x563a5e5df180 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x563a5e67bf20 tx=0x563a5e74e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1dfb9ed26d36d46 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbd85d3700 1 -- 172.21.15.167:0/2829589012 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e77c000 con 0x563a5e763400 2024-09-13T23:44:29.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbd85d3700 1 -- 172.21.15.167:0/2829589012 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a5e692600 con 0x563a5e763400 2024-09-13T23:44:29.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.441+0000 7fcbd85d3700 1 -- 172.21.15.167:0/2829589012 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e77c160 con 0x563a5e763400 2024-09-13T23:44:29.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 -- 172.21.15.167:0/2829589012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e763400 msgr2=0x563a5e5df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 --2- 172.21.15.167:0/2829589012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e763400 0x563a5e5df180 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x563a5e67bf20 tx=0x563a5e74e060 comp rx=0 tx=0).stop 2024-09-13T23:44:29.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 -- 172.21.15.167:0/2829589012 shutdown_connections 2024-09-13T23:44:29.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 --2- 172.21.15.167:0/2829589012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e763400 0x563a5e5df180 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 -- 172.21.15.167:0/2829589012 >> 172.21.15.167:0/2829589012 conn(0x563a5e762800 msgr2=0x563a5e3f8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 -- 172.21.15.167:0/2829589012 shutdown_connections 2024-09-13T23:44:29.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 -- 172.21.15.167:0/2829589012 wait complete. 2024-09-13T23:44:29.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 Processor -- start 2024-09-13T23:44:29.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 -- start start 2024-09-13T23:44:29.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e762400 0x563a5e5df700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5e384000 con 0x563a5e762400 2024-09-13T23:44:29.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf1605700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e762400 0x563a5e5df700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf1605700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e762400 0x563a5e5df700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42666/0 (socket says 172.21.15.167:42666) 2024-09-13T23:44:29.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf1605700 1 -- 172.21.15.167:0/148959040 learned_addr learned my addr 172.21.15.167:0/148959040 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf1605700 1 -- 172.21.15.167:0/148959040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a5e308d00 con 0x563a5e762400 2024-09-13T23:44:29.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf1605700 1 --2- 172.21.15.167:0/148959040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e762400 0x563a5e5df700 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x563a5e77e150 tx=0x563a5e74e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbd6dd0700 1 -- 172.21.15.167:0/148959040 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e77c2c0 con 0x563a5e762400 2024-09-13T23:44:29.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 -- 172.21.15.167:0/148959040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563a5e308340 con 0x563a5e762400 2024-09-13T23:44:29.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbd6dd0700 1 -- 172.21.15.167:0/148959040 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a5e692300 con 0x563a5e762400 2024-09-13T23:44:29.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbd6dd0700 1 -- 172.21.15.167:0/148959040 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x563a5e77c000 con 0x563a5e762400 2024-09-13T23:44:29.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.445+0000 7fcbf364df40 1 -- 172.21.15.167:0/148959040 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a5e308b60 con 0x563a5e762400 2024-09-13T23:44:29.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbd6dd0700 1 -- 172.21.15.167:0/148959040 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x563a5e3f1b80 con 0x563a5e762400 2024-09-13T23:44:29.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbd6dd0700 1 --2- 172.21.15.167:0/148959040 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e7f8000 0x563a5e7fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbd6dd0700 1 -- 172.21.15.167:0/148959040 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x563a5e35d500 con 0x563a5e762400 2024-09-13T23:44:29.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbf364df40 5 note: GC not initialized 2024-09-13T23:44:29.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbf2607700 1 --2- 172.21.15.167:0/148959040 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e7f8000 0x563a5e7fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbf2607700 1 --2- 172.21.15.167:0/148959040 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e7f8000 0x563a5e7fe000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x563a5e67a1b0 tx=0x563a5e661500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbd45cb700 20 reqs_thread_entry: start 2024-09-13T23:44:29.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbf364df40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:29.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbf364df40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:29.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbd1dc6700 20 reqs_thread_entry: start 2024-09-13T23:44:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbf364df40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e7f8400 con 0x563a5e6a3000 2024-09-13T23:44:29.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.449+0000 7fcbd25c7700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:59 2024 2024-09-13T23:44:29.458 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.453+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.453+0000 7fcbf364df40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:44:29.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.453+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e7f8800 con 0x563a5e6a3800 2024-09-13T23:44:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.453+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.453+0000 7fcbd8dd4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:29.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.453+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94808692932608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e7f8c00 con 0x563a5e6a3000 2024-09-13T23:44:29.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.457+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94808692932608] v35'205 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.457+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94808691260160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e7f9000 con 0x563a5e6a3400 2024-09-13T23:44:29.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.461+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94808691260160] v35'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.461+0000 7fcbf364df40 20 remove_watcher() i=0 2024-09-13T23:44:29.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.461+0000 7fcbf364df40 2 removed watcher, disabling cache 2024-09-13T23:44:29.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.461+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94808691261312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e7f9400 con 0x563a5e6a3000 2024-09-13T23:44:29.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.461+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94808691261312] v35'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3000 2024-09-13T23:44:29.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.461+0000 7fcbf364df40 20 remove_watcher() i=1 2024-09-13T23:44:29.467 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.461+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94808691262464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e7f9800 con 0x563a5e6a3400 2024-09-13T23:44:29.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.465+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94808691262464] v35'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.465+0000 7fcbf364df40 20 remove_watcher() i=2 2024-09-13T23:44:29.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.465+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94808691263616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e7f9c00 con 0x563a5e6a3c00 2024-09-13T23:44:29.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.465+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94808691263616] v35'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.465+0000 7fcbf364df40 20 remove_watcher() i=3 2024-09-13T23:44:29.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.465+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94808691264768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e6c6000 con 0x563a5e6a3400 2024-09-13T23:44:29.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.469+0000 7fcbf1e06700 1 -- 172.21.15.167:0/2891422129 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94808691264768] v35'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d200 con 0x563a5e6a3400 2024-09-13T23:44:29.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.469+0000 7fcbf364df40 20 remove_watcher() i=4 2024-09-13T23:44:29.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.469+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94808691265920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e6c6400 con 0x563a5e6a3800 2024-09-13T23:44:29.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.473+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94808691265920] v35'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.473+0000 7fcbf364df40 20 remove_watcher() i=5 2024-09-13T23:44:29.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.473+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94808691267072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e6c6800 con 0x563a5e6a3800 2024-09-13T23:44:29.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.473+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94808691267072] v35'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71d440 con 0x563a5e6a3800 2024-09-13T23:44:29.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.473+0000 7fcbf364df40 20 remove_watcher() i=6 2024-09-13T23:44:29.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.473+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94808691268224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x563a5e6c6c00 con 0x563a5e6a3c00 2024-09-13T23:44:29.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf2607700 1 -- 172.21.15.167:0/2891422129 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94808691268224] v35'203 uv201 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x563a5e71cb40 con 0x563a5e6a3c00 2024-09-13T23:44:29.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 20 remove_watcher() i=7 2024-09-13T23:44:29.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x563a5e6a3400 msgr2=0x563a5e5ddb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x563a5e6a3400 0x563a5e5ddb80 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563a5e6a3000 msgr2=0x563a5e5dd600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x563a5e6a3000 0x563a5e5dd600 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x563a5e6a3c00 msgr2=0x563a5e5dc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x563a5e6a3c00 0x563a5e5dc580 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x563a5e6a3800 msgr2=0x563a5e5dec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x563a5e6a3800 0x563a5e5dec00 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e6a2c00 msgr2=0x563a5e5de680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e6a2c00 0x563a5e5de680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x563a5f0c0030 tx=0x563a5e74e0c0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 msgr2=0x563a5e5dd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 0x563a5e5dd080 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x563a5f0c1e60 tx=0x563a5e3fbbc0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf1605700 1 -- 172.21.15.167:0/2891422129 reap_dead start 2024-09-13T23:44:29.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 shutdown_connections 2024-09-13T23:44:29.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 --2- 172.21.15.167:0/2891422129 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7c00 0x563a5e5dd080 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 >> 172.21.15.167:0/2891422129 conn(0x563a5e2f7800 msgr2=0x563a5e3f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 shutdown_connections 2024-09-13T23:44:29.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 -- 172.21.15.167:0/2891422129 wait complete. 2024-09-13T23:44:29.487 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 -- 172.21.15.167:0/148959040 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e7f8000 msgr2=0x563a5e7fe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 --2- 172.21.15.167:0/148959040 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e7f8000 0x563a5e7fe000 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x563a5e67a1b0 tx=0x563a5e661500 comp rx=0 tx=0).stop 2024-09-13T23:44:29.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 -- 172.21.15.167:0/148959040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e762400 msgr2=0x563a5e5df700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.477+0000 7fcbf364df40 1 --2- 172.21.15.167:0/148959040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e762400 0x563a5e5df700 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x563a5e77e150 tx=0x563a5e74e180 comp rx=0 tx=0).stop 2024-09-13T23:44:29.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 -- 172.21.15.167:0/148959040 shutdown_connections 2024-09-13T23:44:29.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 --2- 172.21.15.167:0/148959040 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e7f8000 0x563a5e7fe000 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 --2- 172.21.15.167:0/148959040 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e762400 0x563a5e5df700 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 -- 172.21.15.167:0/148959040 >> 172.21.15.167:0/148959040 conn(0x563a5e762800 msgr2=0x563a5e3f8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 -- 172.21.15.167:0/148959040 shutdown_connections 2024-09-13T23:44:29.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 -- 172.21.15.167:0/148959040 wait complete. 2024-09-13T23:44:29.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 -- 172.21.15.167:0/3574996295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e6a2000 msgr2=0x563a5e5de100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 --2- 172.21.15.167:0/3574996295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e6a2000 0x563a5e5de100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x563a5e61f650 tx=0x563a5e660c60 comp rx=0 tx=0).stop 2024-09-13T23:44:29.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 -- 172.21.15.167:0/3574996295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7400 msgr2=0x563a5e5dcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 --2- 172.21.15.167:0/3574996295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7400 0x563a5e5dcb00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x563a5f0c1e90 tx=0x563a5e3fb920 comp rx=0 tx=0).stop 2024-09-13T23:44:29.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 -- 172.21.15.167:0/3574996295 shutdown_connections 2024-09-13T23:44:29.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 --2- 172.21.15.167:0/3574996295 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x563a5e6a2000 0x563a5e5de100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 --2- 172.21.15.167:0/3574996295 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x563a5e2f7400 0x563a5e5dcb00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 -- 172.21.15.167:0/3574996295 >> 172.21.15.167:0/3574996295 conn(0x563a5e2f6c00 msgr2=0x563a5e3f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 -- 172.21.15.167:0/3574996295 shutdown_connections 2024-09-13T23:44:29.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.481+0000 7fcbf364df40 1 -- 172.21.15.167:0/3574996295 wait complete. 2024-09-13T23:44:29.490 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "entries": [ 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "buckets": [ 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "", 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.491 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo", 2024-09-13T23:44:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "put_obj", 2024-09-13T23:44:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 30000042, 2024-09-13T23:44:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 11, 2024-09-13T23:44:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 11 2024-09-13T23:44:29.492 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo2", 2024-09-13T23:44:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.493 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo3", 2024-09-13T23:44:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo4", 2024-09-13T23:44:29.494 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "summary": [ 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.495 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "put_obj", 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 30000042, 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 11, 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 11 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "total": { 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 30000042, 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 11, 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 11 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.496 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.497 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.497 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:44:29.498 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}}]} 2024-09-13T23:44:29.498 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-09-13T23:44:29.498 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-09-13T23:44:29.499 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_obj 2024-09-13T23:44:29.541 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:29.541 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:29.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.537+0000 7f3698cfff40 1 Processor -- start 2024-09-13T23:44:29.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.537+0000 7f3698cfff40 1 -- start start 2024-09-13T23:44:29.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.537+0000 7f3698cfff40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5579f1e5b400 0x5579f2140580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.537+0000 7f3698cfff40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5579f1f1f500 con 0x5579f1e5b000 2024-09-13T23:44:29.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.537+0000 7f3698cfff40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5579f1ee8a00 con 0x5579f1e5b400 2024-09-13T23:44:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.537+0000 7f3696cb7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5579f1e5b400 0x5579f2140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.537+0000 7f36974b8700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5579f1e5b000 0x5579f1f5a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:36406/0 (socket says 172.21.15.167:36406) 2024-09-13T23:44:29.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.537+0000 7f36974b8700 1 -- 172.21.15.167:0/591495523 learned_addr learned my addr 172.21.15.167:0/591495523 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:44:29.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.537+0000 7f3696cb7700 1 -- 172.21.15.167:0/591495523 >> v1:172.21.15.12:6789/0 conn(0x5579f1e5b000 legacy=0x5579f1f5a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.537+0000 7f3696cb7700 1 -- 172.21.15.167:0/591495523 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579f1e6dd40 con 0x5579f1e5b400 2024-09-13T23:44:29.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f3696cb7700 1 --2- 172.21.15.167:0/591495523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b400 0x5579f2140580 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x5579f213b920 tx=0x5579f1f5f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16ab255c21d0692 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f36964b6700 1 -- 172.21.15.167:0/591495523 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f1eff4a0 con 0x5579f1e5b400 2024-09-13T23:44:29.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f36964b6700 1 -- 172.21.15.167:0/591495523 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5579f1f1f800 con 0x5579f1e5b400 2024-09-13T23:44:29.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f36964b6700 1 -- 172.21.15.167:0/591495523 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f1eff600 con 0x5579f1e5b400 2024-09-13T23:44:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f3698cfff40 1 -- 172.21.15.167:0/591495523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b400 msgr2=0x5579f2140580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f3698cfff40 1 --2- 172.21.15.167:0/591495523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b400 0x5579f2140580 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x5579f213b920 tx=0x5579f1f5f920 comp rx=0 tx=0).stop 2024-09-13T23:44:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f3698cfff40 1 -- 172.21.15.167:0/591495523 shutdown_connections 2024-09-13T23:44:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f3698cfff40 1 --2- 172.21.15.167:0/591495523 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b400 0x5579f2140580 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f3698cfff40 1 -- 172.21.15.167:0/591495523 >> 172.21.15.167:0/591495523 conn(0x5579f1e5ac00 msgr2=0x5579f1f5a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f3698cfff40 1 -- 172.21.15.167:0/591495523 shutdown_connections 2024-09-13T23:44:29.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f3698cfff40 1 -- 172.21.15.167:0/591495523 wait complete. 2024-09-13T23:44:29.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f3698cfff40 1 Processor -- start 2024-09-13T23:44:29.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f3698cfff40 1 -- start start 2024-09-13T23:44:29.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f3698cfff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b000 0x5579f2140580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f3698cfff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f1ee8500 con 0x5579f1e5b000 2024-09-13T23:44:29.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f36974b8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b000 0x5579f2140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f36974b8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b000 0x5579f2140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42678/0 (socket says 172.21.15.167:42678) 2024-09-13T23:44:29.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.541+0000 7f36974b8700 1 -- 172.21.15.167:0/4174917454 learned_addr learned my addr 172.21.15.167:0/4174917454 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.549 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f36974b8700 1 -- 172.21.15.167:0/4174917454 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579f1e6dba0 con 0x5579f1e5b000 2024-09-13T23:44:29.550 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f36974b8700 1 --2- 172.21.15.167:0/4174917454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b000 0x5579f2140580 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x5579f1fb7e60 tx=0x5579f1f5fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3496d05be03807d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3695cb5700 1 -- 172.21.15.167:0/4174917454 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f1eff1e0 con 0x5579f1e5b000 2024-09-13T23:44:29.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3695cb5700 1 -- 172.21.15.167:0/4174917454 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5579f1f1fb00 con 0x5579f1e5b000 2024-09-13T23:44:29.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3695cb5700 1 -- 172.21.15.167:0/4174917454 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f1eff4a0 con 0x5579f1e5b000 2024-09-13T23:44:29.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3698cfff40 1 -- 172.21.15.167:0/4174917454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b000 msgr2=0x5579f2140580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3698cfff40 1 --2- 172.21.15.167:0/4174917454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b000 0x5579f2140580 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x5579f1fb7e60 tx=0x5579f1f5fb60 comp rx=0 tx=0).stop 2024-09-13T23:44:29.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3698cfff40 1 -- 172.21.15.167:0/4174917454 shutdown_connections 2024-09-13T23:44:29.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3698cfff40 1 --2- 172.21.15.167:0/4174917454 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b000 0x5579f2140580 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3698cfff40 1 -- 172.21.15.167:0/4174917454 >> 172.21.15.167:0/4174917454 conn(0x5579f1e5ac00 msgr2=0x5579f1f5b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.551 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3698cfff40 1 -- 172.21.15.167:0/4174917454 shutdown_connections 2024-09-13T23:44:29.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3698cfff40 1 -- 172.21.15.167:0/4174917454 wait complete. 2024-09-13T23:44:29.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3698cfff40 1 Processor -- start 2024-09-13T23:44:29.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3698cfff40 1 -- start start 2024-09-13T23:44:29.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3698cfff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b400 0x5579f2140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f3698cfff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f1ee83c0 con 0x5579f1e5b400 2024-09-13T23:44:29.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f36974b8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b400 0x5579f2140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f36974b8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b400 0x5579f2140b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42684/0 (socket says 172.21.15.167:42684) 2024-09-13T23:44:29.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f36974b8700 1 -- 172.21.15.167:0/1932894082 learned_addr learned my addr 172.21.15.167:0/1932894082 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.545+0000 7f36974b8700 1 -- 172.21.15.167:0/1932894082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579f1e6da00 con 0x5579f1e5b400 2024-09-13T23:44:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f36974b8700 1 --2- 172.21.15.167:0/1932894082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b400 0x5579f2140b00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x5579f1fb7710 tx=0x5579f1f5fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f36944b2700 1 -- 172.21.15.167:0/1932894082 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f1eff600 con 0x5579f1e5b400 2024-09-13T23:44:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f3698cfff40 1 -- 172.21.15.167:0/1932894082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5579f1e6d860 con 0x5579f1e5b400 2024-09-13T23:44:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f36944b2700 1 -- 172.21.15.167:0/1932894082 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5579f1f1fc80 con 0x5579f1e5b400 2024-09-13T23:44:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f36944b2700 1 -- 172.21.15.167:0/1932894082 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f1eff1e0 con 0x5579f1e5b400 2024-09-13T23:44:29.554 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f3698cfff40 1 -- 172.21.15.167:0/1932894082 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579f1e6d6c0 con 0x5579f1e5b400 2024-09-13T23:44:29.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f3698cfff40 1 Processor -- start 2024-09-13T23:44:29.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f3698cfff40 1 -- start start 2024-09-13T23:44:29.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f36944b2700 1 -- 172.21.15.167:0/1932894082 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5579f1f54b00 con 0x5579f1e5b400 2024-09-13T23:44:29.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f3698cfff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 0x5579f2141080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f3698cfff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f1ee8000 con 0x5579f1e5bc00 2024-09-13T23:44:29.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f36944b2700 1 --2- 172.21.15.167:0/1932894082 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2214000 0x5579f2142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f3697cb9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 0x5579f2141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f36944b2700 1 -- 172.21.15.167:0/1932894082 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5579f1ec1180 con 0x5579f1e5b400 2024-09-13T23:44:29.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f3697cb9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 0x5579f2141080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42694/0 (socket says 172.21.15.167:42694) 2024-09-13T23:44:29.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f3697cb9700 1 -- 172.21.15.167:0/2602613628 learned_addr learned my addr 172.21.15.167:0/2602613628 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f36974b8700 1 --2- 172.21.15.167:0/1932894082 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2214000 0x5579f2142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f3697cb9700 1 -- 172.21.15.167:0/2602613628 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579f1e6c000 con 0x5579f1e5bc00 2024-09-13T23:44:29.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.549+0000 7f36974b8700 1 --2- 172.21.15.167:0/1932894082 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2214000 0x5579f2142100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5579f2191530 tx=0x5579f21c4240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3697cb9700 1 --2- 172.21.15.167:0/2602613628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 0x5579f2141080 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x5579f1fb60c0 tx=0x5579f21c50e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8808a2cfbe38ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3691cad700 1 -- 172.21.15.167:0/2602613628 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f1eff600 con 0x5579f1e5bc00 2024-09-13T23:44:29.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3691cad700 1 -- 172.21.15.167:0/2602613628 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5579f2226000 con 0x5579f1e5bc00 2024-09-13T23:44:29.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3691cad700 1 -- 172.21.15.167:0/2602613628 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f1eff1e0 con 0x5579f1e5bc00 2024-09-13T23:44:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 -- 172.21.15.167:0/2602613628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 msgr2=0x5579f2141080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 --2- 172.21.15.167:0/2602613628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 0x5579f2141080 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x5579f1fb60c0 tx=0x5579f21c50e0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 -- 172.21.15.167:0/2602613628 shutdown_connections 2024-09-13T23:44:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 --2- 172.21.15.167:0/2602613628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 0x5579f2141080 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 -- 172.21.15.167:0/2602613628 >> 172.21.15.167:0/2602613628 conn(0x5579f1e5b800 msgr2=0x5579f1f5b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 -- 172.21.15.167:0/2602613628 shutdown_connections 2024-09-13T23:44:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 -- 172.21.15.167:0/2602613628 wait complete. 2024-09-13T23:44:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 Processor -- start 2024-09-13T23:44:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 -- start start 2024-09-13T23:44:29.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 0x5579f2141080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f1e80640 con 0x5579f1e5bc00 2024-09-13T23:44:29.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3697cb9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 0x5579f2141080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3697cb9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 0x5579f2141080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42696/0 (socket says 172.21.15.167:42696) 2024-09-13T23:44:29.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 learned_addr learned my addr 172.21.15.167:0/763752225 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579f1e6cea0 con 0x5579f1e5bc00 2024-09-13T23:44:29.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3697cb9700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 0x5579f2141080 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x5579f2190840 tx=0x5579f1f5ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f36904aa700 1 -- 172.21.15.167:0/763752225 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f1effa20 con 0x5579f1e5bc00 2024-09-13T23:44:29.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f36904aa700 1 -- 172.21.15.167:0/763752225 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5579f2226180 con 0x5579f1e5bc00 2024-09-13T23:44:29.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5579f1e6c820 con 0x5579f1e5bc00 2024-09-13T23:44:29.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f36904aa700 1 -- 172.21.15.167:0/763752225 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f1eff600 con 0x5579f1e5bc00 2024-09-13T23:44:29.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.553+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579f1e6c9c0 con 0x5579f1e5bc00 2024-09-13T23:44:29.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f36904aa700 1 -- 172.21.15.167:0/763752225 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5579f1f54dc0 con 0x5579f1e5bc00 2024-09-13T23:44:29.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f36904aa700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2214c00 0x5579f2142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f36904aa700 1 -- 172.21.15.167:0/763752225 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5579f1ec1340 con 0x5579f1e5bc00 2024-09-13T23:44:29.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f3696cb7700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2214c00 0x5579f2142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f3698cfff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f3698cfff40 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5579f2215000 0x5579f2141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2215400 con 0x5579f2215000 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f3697cb9700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5579f2215000 0x5579f2141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f3696cb7700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2214c00 0x5579f2142680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5579f213b920 tx=0x5579f1f5f800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f3697cb9700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5579f2215000 0x5579f2141600 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f3698cfff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f3698cfff40 20 realm 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f3698cfff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.557+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2215800 con 0x5579f2215000 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:29.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2215c00 con 0x5579f2215000 2024-09-13T23:44:29.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5579f2215400 0x5579f2141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2215800 con 0x5579f2215400 2024-09-13T23:44:29.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f36974b8700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5579f2215400 0x5579f2141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f36974b8700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5579f2215400 0x5579f2141b80 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:29.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2215c00 con 0x5579f2215400 2024-09-13T23:44:29.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:29.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 20 searching for the correct realm 2024-09-13T23:44:29.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5579f2215800 0x5579f2142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.561+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2215c00 con 0x5579f2215800 2024-09-13T23:44:29.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.565+0000 7f3696cb7700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5579f2215800 0x5579f2142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.565+0000 7f3696cb7700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5579f2215800 0x5579f2142c00 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.565+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.565+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2215c00 con 0x5579f2215400 2024-09-13T23:44:29.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.565+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.565+0000 7f36974b8700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5579f2215c00 0x5579f2140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.565+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215c00 2024-09-13T23:44:29.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.565+0000 7f3697cb9700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5579f2215c00 0x5579f2140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.565+0000 7f3697cb9700 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5579f2215c00 0x5579f2140580 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.565+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.565+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215000 2024-09-13T23:44:29.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215000 2024-09-13T23:44:29.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215800 2024-09-13T23:44:29.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215000 2024-09-13T23:44:29.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215400 2024-09-13T23:44:29.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215400 2024-09-13T23:44:29.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215000 2024-09-13T23:44:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.569+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215c00 2024-09-13T23:44:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.573+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.573+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215000 2024-09-13T23:44:29.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.573+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.573+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215c00 2024-09-13T23:44:29.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.573+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.573+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215400 2024-09-13T23:44:29.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.573+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.573+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215800 2024-09-13T23:44:29.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.573+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.573+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215000 2024-09-13T23:44:29.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.573+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.573+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215400 2024-09-13T23:44:29.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.577+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.577+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215800 2024-09-13T23:44:29.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.577+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.577+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215000 2024-09-13T23:44:29.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.577+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.577+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215c00 2024-09-13T23:44:29.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.577+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.577+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215000 2024-09-13T23:44:29.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.577+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.577+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215800 2024-09-13T23:44:29.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.577+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.577+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215c00 2024-09-13T23:44:29.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.581+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.581+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215000 2024-09-13T23:44:29.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.581+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.581+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215c00 2024-09-13T23:44:29.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.581+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.581+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215800 2024-09-13T23:44:29.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.581+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.581+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215c00 2024-09-13T23:44:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.581+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.581+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215400 2024-09-13T23:44:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.581+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215400 2024-09-13T23:44:29.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215800 2024-09-13T23:44:29.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215400 2024-09-13T23:44:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215400 2024-09-13T23:44:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f3698cfff40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f3698cfff40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f3698cfff40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f3698cfff40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f3698cfff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.585+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2800 con 0x5579f2215000 2024-09-13T23:44:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2c00 con 0x5579f2215400 2024-09-13T23:44:29.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:29.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3000 con 0x5579f2215400 2024-09-13T23:44:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 20 zone default found 2024-09-13T23:44:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 4 Realm: () 2024-09-13T23:44:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:29.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 20 zonegroup default 2024-09-13T23:44:29.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3400 con 0x5579f2215c00 2024-09-13T23:44:29.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.589+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3800 con 0x5579f2215800 2024-09-13T23:44:29.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.593+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.593+0000 7f3698cfff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.593+0000 7f3698cfff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.593+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3c00 con 0x5579f2215800 2024-09-13T23:44:29.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.593+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.593+0000 7f3698cfff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.593+0000 7f3698cfff40 20 started sync module instance, tier type = 2024-09-13T23:44:29.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.593+0000 7f3698cfff40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:29.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.593+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3800 con 0x5579f2215400 2024-09-13T23:44:29.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.597+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.597+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93982240766720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3c00 con 0x5579f2215400 2024-09-13T23:44:29.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.597+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2400 con 0x5579f2215000 2024-09-13T23:44:29.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.597+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.601+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93982240767872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215000 2024-09-13T23:44:29.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.601+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2c00 con 0x5579f2215400 2024-09-13T23:44:29.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.601+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 93982240767872] v35'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.609 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.605+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 93982240766720] v35'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.605+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.605+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93982240769024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3000 con 0x5579f2215400 2024-09-13T23:44:29.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.605+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2800 con 0x5579f2215c00 2024-09-13T23:44:29.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.609+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.609+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93982240770176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3400 con 0x5579f2215c00 2024-09-13T23:44:29.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.609+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2400 con 0x5579f2215400 2024-09-13T23:44:29.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.609+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 93982240769024] v35'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.613+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93982240770176] v35'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.613+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.613+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93982240771328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2000 con 0x5579f2215400 2024-09-13T23:44:29.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.613+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3800 con 0x5579f2215800 2024-09-13T23:44:29.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.613+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 93982240771328] v35'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.613+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.613+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93982240772480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3c00 con 0x5579f2215800 2024-09-13T23:44:29.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.613+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2c00 con 0x5579f2215800 2024-09-13T23:44:29.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.617+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v35'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.617+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93982240773632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3000 con 0x5579f2215800 2024-09-13T23:44:29.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.617+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2400 con 0x5579f2215c00 2024-09-13T23:44:29.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.617+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 93982240772480] v35'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 93982240773632] v35'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'204 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93982240774784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3800 con 0x5579f2215c00 2024-09-13T23:44:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3698cfff40 20 add_watcher() i=0 2024-09-13T23:44:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3698cfff40 20 add_watcher() i=1 2024-09-13T23:44:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3698cfff40 20 add_watcher() i=2 2024-09-13T23:44:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3698cfff40 20 add_watcher() i=3 2024-09-13T23:44:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3698cfff40 20 add_watcher() i=4 2024-09-13T23:44:29.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3698cfff40 20 add_watcher() i=5 2024-09-13T23:44:29.626 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3698cfff40 20 add_watcher() i=6 2024-09-13T23:44:29.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 93982240774784] v35'205 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3698cfff40 20 add_watcher() i=7 2024-09-13T23:44:29.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3698cfff40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:29.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3c00 con 0x5579f2215000 2024-09-13T23:44:29.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.628 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.621+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93982242512896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d2c00 con 0x5579f2215000 2024-09-13T23:44:29.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.625+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93982242512896] v35'206 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.625+0000 7f367e486700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:29.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.625+0000 7f3698cfff40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:29.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.625+0000 7f3698cfff40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:29.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.625+0000 7f3698cfff40 1 Processor -- start 2024-09-13T23:44:29.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.625+0000 7f3698cfff40 1 -- start start 2024-09-13T23:44:29.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.625+0000 7f3698cfff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 0x5579f2143180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.625+0000 7f3698cfff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f1e80780 con 0x5579f22d3400 2024-09-13T23:44:29.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.625+0000 7f3696cb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 0x5579f2143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.625+0000 7f3696cb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 0x5579f2143180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42698/0 (socket says 172.21.15.167:42698) 2024-09-13T23:44:29.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.625+0000 7f3696cb7700 1 -- 172.21.15.167:0/398878656 learned_addr learned my addr 172.21.15.167:0/398878656 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.625+0000 7f3696cb7700 1 -- 172.21.15.167:0/398878656 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579f1e6cb60 con 0x5579f22d3400 2024-09-13T23:44:29.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3696cb7700 1 --2- 172.21.15.167:0/398878656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 0x5579f2143180 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x5579f22813e0 tx=0x5579f1f5f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba3e65e7c48824 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f367dc85700 1 -- 172.21.15.167:0/398878656 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f1eff760 con 0x5579f22d3400 2024-09-13T23:44:29.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f367dc85700 1 -- 172.21.15.167:0/398878656 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5579f2226600 con 0x5579f22d3400 2024-09-13T23:44:29.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f367dc85700 1 -- 172.21.15.167:0/398878656 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f22f2000 con 0x5579f22d3400 2024-09-13T23:44:29.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3698cfff40 1 -- 172.21.15.167:0/398878656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 msgr2=0x5579f2143180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3698cfff40 1 --2- 172.21.15.167:0/398878656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 0x5579f2143180 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x5579f22813e0 tx=0x5579f1f5f860 comp rx=0 tx=0).stop 2024-09-13T23:44:29.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3698cfff40 1 -- 172.21.15.167:0/398878656 shutdown_connections 2024-09-13T23:44:29.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3698cfff40 1 --2- 172.21.15.167:0/398878656 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 0x5579f2143180 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3698cfff40 1 -- 172.21.15.167:0/398878656 >> 172.21.15.167:0/398878656 conn(0x5579f22d2800 msgr2=0x5579f1f5c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3698cfff40 1 -- 172.21.15.167:0/398878656 shutdown_connections 2024-09-13T23:44:29.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3698cfff40 1 -- 172.21.15.167:0/398878656 wait complete. 2024-09-13T23:44:29.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3698cfff40 1 Processor -- start 2024-09-13T23:44:29.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3698cfff40 1 -- start start 2024-09-13T23:44:29.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3698cfff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 0x5579f2143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3698cfff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579f1ee8000 con 0x5579f22d3400 2024-09-13T23:44:29.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3696cb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 0x5579f2143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3696cb7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 0x5579f2143180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42714/0 (socket says 172.21.15.167:42714) 2024-09-13T23:44:29.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3696cb7700 1 -- 172.21.15.167:0/1873445797 learned_addr learned my addr 172.21.15.167:0/1873445797 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3696cb7700 1 -- 172.21.15.167:0/1873445797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579f1e6cd00 con 0x5579f22d3400 2024-09-13T23:44:29.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.629+0000 7f3696cb7700 1 --2- 172.21.15.167:0/1873445797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 0x5579f2143180 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x5579f213bf80 tx=0x5579f22b6420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f367c482700 1 -- 172.21.15.167:0/1873445797 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f22f2160 con 0x5579f22d3400 2024-09-13T23:44:29.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f3698cfff40 1 -- 172.21.15.167:0/1873445797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5579f1e6c340 con 0x5579f22d3400 2024-09-13T23:44:29.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f367c482700 1 -- 172.21.15.167:0/1873445797 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5579f2226300 con 0x5579f22d3400 2024-09-13T23:44:29.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f367c482700 1 -- 172.21.15.167:0/1873445797 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5579f22f22c0 con 0x5579f22d3400 2024-09-13T23:44:29.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f3698cfff40 1 -- 172.21.15.167:0/1873445797 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5579f1e6dd40 con 0x5579f22d3400 2024-09-13T23:44:29.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f367c482700 1 -- 172.21.15.167:0/1873445797 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5579f1f55b80 con 0x5579f22d3400 2024-09-13T23:44:29.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f367c482700 1 --2- 172.21.15.167:0/1873445797 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2366000 0x5579f236c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f367c482700 1 -- 172.21.15.167:0/1873445797 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5579f1ec1500 con 0x5579f22d3400 2024-09-13T23:44:29.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f3698cfff40 5 note: GC not initialized 2024-09-13T23:44:29.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f3697cb9700 1 --2- 172.21.15.167:0/1873445797 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2366000 0x5579f236c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f3697cb9700 1 --2- 172.21.15.167:0/1873445797 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2366000 0x5579f236c000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x5579f2281ce0 tx=0x5579f21c57a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f3679c7d700 20 reqs_thread_entry: start 2024-09-13T23:44:29.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f3698cfff40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:29.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f3698cfff40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:29.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.633+0000 7f3677c79700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:44:59 2024 2024-09-13T23:44:29.641 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:29.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.637+0000 7f3677457700 20 reqs_thread_entry: start 2024-09-13T23:44:29.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.637+0000 7f3698cfff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.637+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f22d3c00 con 0x5579f2215000 2024-09-13T23:44:29.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.637+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.637+0000 7f3698cfff40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:44:29.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.637+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2366400 con 0x5579f2215800 2024-09-13T23:44:29.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.637+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.637+0000 7f367e486700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:29.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.637+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93982242512896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2366800 con 0x5579f2215000 2024-09-13T23:44:29.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.641+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 93982242512896] v35'207 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.641+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93982240766720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2366c00 con 0x5579f2215400 2024-09-13T23:44:29.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.645+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 93982240766720] v35'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.645+0000 7f3698cfff40 20 remove_watcher() i=0 2024-09-13T23:44:29.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.645+0000 7f3698cfff40 2 removed watcher, disabling cache 2024-09-13T23:44:29.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.645+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93982240767872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2367000 con 0x5579f2215000 2024-09-13T23:44:29.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.645+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 93982240767872] v35'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215000 2024-09-13T23:44:29.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.645+0000 7f3698cfff40 20 remove_watcher() i=1 2024-09-13T23:44:29.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.645+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93982240769024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2367400 con 0x5579f2215400 2024-09-13T23:44:29.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.649+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 93982240769024] v35'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.649+0000 7f3698cfff40 20 remove_watcher() i=2 2024-09-13T23:44:29.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.649+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93982240770176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2367800 con 0x5579f2215c00 2024-09-13T23:44:29.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.653+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 93982240770176] v35'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.653+0000 7f3698cfff40 20 remove_watcher() i=3 2024-09-13T23:44:29.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.653+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93982240771328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f2367c00 con 0x5579f2215400 2024-09-13T23:44:29.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.653+0000 7f36974b8700 1 -- 172.21.15.167:0/763752225 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 93982240771328] v35'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9200 con 0x5579f2215400 2024-09-13T23:44:29.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.653+0000 7f3698cfff40 20 remove_watcher() i=4 2024-09-13T23:44:29.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.653+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93982240772480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f224a000 con 0x5579f2215800 2024-09-13T23:44:29.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.657+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 93982240772480] v35'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.657+0000 7f3698cfff40 20 remove_watcher() i=5 2024-09-13T23:44:29.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.657+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93982240773632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f224a400 con 0x5579f2215800 2024-09-13T23:44:29.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 93982240773632] v35'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c9440 con 0x5579f2215800 2024-09-13T23:44:29.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 20 remove_watcher() i=6 2024-09-13T23:44:29.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93982240774784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5579f224a800 con 0x5579f2215c00 2024-09-13T23:44:29.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3697cb9700 1 -- 172.21.15.167:0/763752225 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 93982240774784] v35'206 uv204 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5579f22c8b40 con 0x5579f2215c00 2024-09-13T23:44:29.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 20 remove_watcher() i=7 2024-09-13T23:44:29.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5579f2215400 msgr2=0x5579f2141b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5579f2215400 0x5579f2141b80 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5579f2215000 msgr2=0x5579f2141600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5579f2215000 0x5579f2141600 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5579f2215c00 msgr2=0x5579f2140580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5579f2215c00 0x5579f2140580 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5579f2215800 msgr2=0x5579f2142c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5579f2215800 0x5579f2142c00 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2214c00 msgr2=0x5579f2142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2214c00 0x5579f2142680 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5579f213b920 tx=0x5579f1f5f800 comp rx=0 tx=0).stop 2024-09-13T23:44:29.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 msgr2=0x5579f2141080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 0x5579f2141080 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x5579f2190840 tx=0x5579f1f5ff20 comp rx=0 tx=0).stop 2024-09-13T23:44:29.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3696cb7700 1 -- 172.21.15.167:0/763752225 reap_dead start 2024-09-13T23:44:29.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 shutdown_connections 2024-09-13T23:44:29.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 --2- 172.21.15.167:0/763752225 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5bc00 0x5579f2141080 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 >> 172.21.15.167:0/763752225 conn(0x5579f1e5b800 msgr2=0x5579f1f5b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 shutdown_connections 2024-09-13T23:44:29.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.661+0000 7f3698cfff40 1 -- 172.21.15.167:0/763752225 wait complete. 2024-09-13T23:44:29.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 -- 172.21.15.167:0/1873445797 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2366000 msgr2=0x5579f236c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 --2- 172.21.15.167:0/1873445797 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2366000 0x5579f236c000 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x5579f2281ce0 tx=0x5579f21c57a0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 -- 172.21.15.167:0/1873445797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 msgr2=0x5579f2143180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 --2- 172.21.15.167:0/1873445797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 0x5579f2143180 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x5579f213bf80 tx=0x5579f22b6420 comp rx=0 tx=0).stop 2024-09-13T23:44:29.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 -- 172.21.15.167:0/1873445797 shutdown_connections 2024-09-13T23:44:29.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 --2- 172.21.15.167:0/1873445797 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2366000 0x5579f236c000 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 --2- 172.21.15.167:0/1873445797 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f22d3400 0x5579f2143180 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 -- 172.21.15.167:0/1873445797 >> 172.21.15.167:0/1873445797 conn(0x5579f22d2800 msgr2=0x5579f1f5c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 -- 172.21.15.167:0/1873445797 shutdown_connections 2024-09-13T23:44:29.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 -- 172.21.15.167:0/1873445797 wait complete. 2024-09-13T23:44:29.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 -- 172.21.15.167:0/1932894082 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2214000 msgr2=0x5579f2142100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 --2- 172.21.15.167:0/1932894082 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2214000 0x5579f2142100 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x5579f2191530 tx=0x5579f21c4240 comp rx=0 tx=0).stop 2024-09-13T23:44:29.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 -- 172.21.15.167:0/1932894082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b400 msgr2=0x5579f2140b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 --2- 172.21.15.167:0/1932894082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b400 0x5579f2140b00 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x5579f1fb7710 tx=0x5579f1f5fa40 comp rx=0 tx=0).stop 2024-09-13T23:44:29.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 -- 172.21.15.167:0/1932894082 shutdown_connections 2024-09-13T23:44:29.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 --2- 172.21.15.167:0/1932894082 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5579f2214000 0x5579f2142100 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 --2- 172.21.15.167:0/1932894082 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5579f1e5b400 0x5579f2140b00 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 -- 172.21.15.167:0/1932894082 >> 172.21.15.167:0/1932894082 conn(0x5579f1e5ac00 msgr2=0x5579f1f5b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 -- 172.21.15.167:0/1932894082 shutdown_connections 2024-09-13T23:44:29.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.665+0000 7f3698cfff40 1 -- 172.21.15.167:0/1932894082 wait complete. 2024-09-13T23:44:29.674 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:44:29.674 INFO:teuthology.orchestra.run.smithi167.stdout: "entries": [ 2024-09-13T23:44:29.674 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.674 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:29.674 INFO:teuthology.orchestra.run.smithi167.stdout: "buckets": [ 2024-09-13T23:44:29.675 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.675 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "", 2024-09-13T23:44:29.675 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.675 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.675 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.675 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:29.675 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.675 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.675 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo", 2024-09-13T23:44:29.675 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.675 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.676 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.676 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.676 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.676 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_obj", 2024-09-13T23:44:29.676 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.676 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.676 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 4, 2024-09-13T23:44:29.676 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:29.676 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.676 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.677 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.677 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.677 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo2", 2024-09-13T23:44:29.677 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.677 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.677 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.677 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:29.677 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.677 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.678 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo3", 2024-09-13T23:44:29.678 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.678 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.678 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.678 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:29.678 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.678 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.679 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo4", 2024-09-13T23:44:29.679 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.679 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.679 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.679 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:29.679 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.679 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.679 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.680 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:29.680 INFO:teuthology.orchestra.run.smithi167.stdout: "summary": [ 2024-09-13T23:44:29.680 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.680 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:29.680 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.680 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.680 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_obj", 2024-09-13T23:44:29.680 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.681 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.681 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 4, 2024-09-13T23:44:29.681 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:29.681 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.681 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:29.681 INFO:teuthology.orchestra.run.smithi167.stdout: "total": { 2024-09-13T23:44:29.681 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.681 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.681 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 4, 2024-09-13T23:44:29.682 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:29.682 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.682 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.682 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.682 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:44:29.683 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}]}, {'bucket': 'myfoo2', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}]} 2024-09-13T23:44:29.683 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-09-13T23:44:29.683 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-09-13T23:44:29.684 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_bucket 2024-09-13T23:44:29.726 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:29.726 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:29.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.721+0000 7fca57041f40 1 Processor -- start 2024-09-13T23:44:29.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.721+0000 7fca57041f40 1 -- start start 2024-09-13T23:44:29.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.721+0000 7fca57041f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f39c455400 0x55f39c73a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.721+0000 7fca57041f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f39c519500 con 0x55f39c455000 2024-09-13T23:44:29.728 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.721+0000 7fca57041f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f39c4e2a00 con 0x55f39c455400 2024-09-13T23:44:29.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.721+0000 7fca54ff9700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f39c455400 0x55f39c73a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.721+0000 7fca54ff9700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f39c455400 0x55f39c73a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42718/0 (socket says 172.21.15.167:42718) 2024-09-13T23:44:29.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.721+0000 7fca54ff9700 1 -- 172.21.15.167:0/1250577905 learned_addr learned my addr 172.21.15.167:0/1250577905 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.721+0000 7fca54ff9700 1 -- 172.21.15.167:0/1250577905 >> v1:172.21.15.12:6789/0 conn(0x55f39c455000 legacy=0x55f39c554800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca54ff9700 1 -- 172.21.15.167:0/1250577905 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f39c467d40 con 0x55f39c455400 2024-09-13T23:44:29.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca54ff9700 1 --2- 172.21.15.167:0/1250577905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455400 0x55f39c73a580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x55f39c735920 tx=0x55f39c559920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b036889dd7af199d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca547f8700 1 -- 172.21.15.167:0/1250577905 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c4f94a0 con 0x55f39c455400 2024-09-13T23:44:29.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca547f8700 1 -- 172.21.15.167:0/1250577905 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f39c519800 con 0x55f39c455400 2024-09-13T23:44:29.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca547f8700 1 -- 172.21.15.167:0/1250577905 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c4f9600 con 0x55f39c455400 2024-09-13T23:44:29.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca57041f40 1 -- 172.21.15.167:0/1250577905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455400 msgr2=0x55f39c73a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca57041f40 1 --2- 172.21.15.167:0/1250577905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455400 0x55f39c73a580 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x55f39c735920 tx=0x55f39c559920 comp rx=0 tx=0).stop 2024-09-13T23:44:29.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca57041f40 1 -- 172.21.15.167:0/1250577905 shutdown_connections 2024-09-13T23:44:29.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca57041f40 1 --2- 172.21.15.167:0/1250577905 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455400 0x55f39c73a580 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca57041f40 1 -- 172.21.15.167:0/1250577905 >> 172.21.15.167:0/1250577905 conn(0x55f39c454c00 msgr2=0x55f39c554000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca57041f40 1 -- 172.21.15.167:0/1250577905 shutdown_connections 2024-09-13T23:44:29.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca57041f40 1 -- 172.21.15.167:0/1250577905 wait complete. 2024-09-13T23:44:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca57041f40 1 Processor -- start 2024-09-13T23:44:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca57041f40 1 -- start start 2024-09-13T23:44:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca57041f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455000 0x55f39c73a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.732 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca57041f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39c4e2500 con 0x55f39c455000 2024-09-13T23:44:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca557fa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455000 0x55f39c73a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca557fa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455000 0x55f39c73a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42728/0 (socket says 172.21.15.167:42728) 2024-09-13T23:44:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca557fa700 1 -- 172.21.15.167:0/3228601710 learned_addr learned my addr 172.21.15.167:0/3228601710 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca557fa700 1 -- 172.21.15.167:0/3228601710 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f39c467ba0 con 0x55f39c455000 2024-09-13T23:44:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.725+0000 7fca557fa700 1 --2- 172.21.15.167:0/3228601710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455000 0x55f39c73a580 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x55f39c5b1e90 tx=0x55f39c559b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c33100d87fa4d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca53ff7700 1 -- 172.21.15.167:0/3228601710 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c4f94a0 con 0x55f39c455000 2024-09-13T23:44:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca53ff7700 1 -- 172.21.15.167:0/3228601710 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f39c519b00 con 0x55f39c455000 2024-09-13T23:44:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca53ff7700 1 -- 172.21.15.167:0/3228601710 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c4f9600 con 0x55f39c455000 2024-09-13T23:44:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 -- 172.21.15.167:0/3228601710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455000 msgr2=0x55f39c73a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.734 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 --2- 172.21.15.167:0/3228601710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455000 0x55f39c73a580 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x55f39c5b1e90 tx=0x55f39c559b60 comp rx=0 tx=0).stop 2024-09-13T23:44:29.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 -- 172.21.15.167:0/3228601710 shutdown_connections 2024-09-13T23:44:29.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 --2- 172.21.15.167:0/3228601710 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455000 0x55f39c73a580 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 -- 172.21.15.167:0/3228601710 >> 172.21.15.167:0/3228601710 conn(0x55f39c454c00 msgr2=0x55f39c555000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 -- 172.21.15.167:0/3228601710 shutdown_connections 2024-09-13T23:44:29.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 -- 172.21.15.167:0/3228601710 wait complete. 2024-09-13T23:44:29.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 Processor -- start 2024-09-13T23:44:29.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 -- start start 2024-09-13T23:44:29.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455400 0x55f39c73ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39c4e23c0 con 0x55f39c455400 2024-09-13T23:44:29.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca557fa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455400 0x55f39c73ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca557fa700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455400 0x55f39c73ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42734/0 (socket says 172.21.15.167:42734) 2024-09-13T23:44:29.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca557fa700 1 -- 172.21.15.167:0/503155591 learned_addr learned my addr 172.21.15.167:0/503155591 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca557fa700 1 -- 172.21.15.167:0/503155591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f39c467a00 con 0x55f39c455400 2024-09-13T23:44:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca557fa700 1 --2- 172.21.15.167:0/503155591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455400 0x55f39c73ab00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x55f39c5b1f80 tx=0x55f39c559a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca527f4700 1 -- 172.21.15.167:0/503155591 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c4f98c0 con 0x55f39c455400 2024-09-13T23:44:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca527f4700 1 -- 172.21.15.167:0/503155591 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f39c519c80 con 0x55f39c455400 2024-09-13T23:44:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 -- 172.21.15.167:0/503155591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f39c467860 con 0x55f39c455400 2024-09-13T23:44:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca527f4700 1 -- 172.21.15.167:0/503155591 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c4f94a0 con 0x55f39c455400 2024-09-13T23:44:29.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.729+0000 7fca57041f40 1 -- 172.21.15.167:0/503155591 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f39c4676c0 con 0x55f39c455400 2024-09-13T23:44:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca57041f40 1 Processor -- start 2024-09-13T23:44:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca57041f40 1 -- start start 2024-09-13T23:44:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca527f4700 1 -- 172.21.15.167:0/503155591 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f39c54eb00 con 0x55f39c455400 2024-09-13T23:44:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca57041f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 0x55f39c73b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca57041f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39c4e2000 con 0x55f39c455c00 2024-09-13T23:44:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca527f4700 1 --2- 172.21.15.167:0/503155591 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c800000 0x55f39c73c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca527f4700 1 -- 172.21.15.167:0/503155591 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f39c4bb180 con 0x55f39c455400 2024-09-13T23:44:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca55ffb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 0x55f39c73b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca55ffb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 0x55f39c73b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42740/0 (socket says 172.21.15.167:42740) 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca55ffb700 1 -- 172.21.15.167:0/1923574422 learned_addr learned my addr 172.21.15.167:0/1923574422 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca557fa700 1 --2- 172.21.15.167:0/503155591 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c800000 0x55f39c73c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca557fa700 1 --2- 172.21.15.167:0/503155591 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c800000 0x55f39c73c100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55f39c76d680 tx=0x55f39c7b4240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca55ffb700 1 -- 172.21.15.167:0/1923574422 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f39c466000 con 0x55f39c455c00 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca55ffb700 1 --2- 172.21.15.167:0/1923574422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 0x55f39c73b080 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x55f39c5b1f20 tx=0x55f39c7b50e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1770db1986bdbb91 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca4ffef700 1 -- 172.21.15.167:0/1923574422 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c4f98c0 con 0x55f39c455c00 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca4ffef700 1 -- 172.21.15.167:0/1923574422 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f39c812000 con 0x55f39c455c00 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca4ffef700 1 -- 172.21.15.167:0/1923574422 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c4f94a0 con 0x55f39c455c00 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca57041f40 1 -- 172.21.15.167:0/1923574422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 msgr2=0x55f39c73b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca57041f40 1 --2- 172.21.15.167:0/1923574422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 0x55f39c73b080 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x55f39c5b1f20 tx=0x55f39c7b50e0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca57041f40 1 -- 172.21.15.167:0/1923574422 shutdown_connections 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca57041f40 1 --2- 172.21.15.167:0/1923574422 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 0x55f39c73b080 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca57041f40 1 -- 172.21.15.167:0/1923574422 >> 172.21.15.167:0/1923574422 conn(0x55f39c455800 msgr2=0x55f39c555800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.733+0000 7fca57041f40 1 -- 172.21.15.167:0/1923574422 shutdown_connections 2024-09-13T23:44:29.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca57041f40 1 -- 172.21.15.167:0/1923574422 wait complete. 2024-09-13T23:44:29.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca57041f40 1 Processor -- start 2024-09-13T23:44:29.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca57041f40 1 -- start start 2024-09-13T23:44:29.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca57041f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 0x55f39c73b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca57041f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39c47a640 con 0x55f39c455c00 2024-09-13T23:44:29.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca55ffb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 0x55f39c73b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca55ffb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 0x55f39c73b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42748/0 (socket says 172.21.15.167:42748) 2024-09-13T23:44:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 learned_addr learned my addr 172.21.15.167:0/1314684254 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f39c466ea0 con 0x55f39c455c00 2024-09-13T23:44:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca55ffb700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 0x55f39c73b080 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x55f39c76c8a0 tx=0x55f39c559f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca4e7ec700 1 -- 172.21.15.167:0/1314684254 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c4f9b80 con 0x55f39c455c00 2024-09-13T23:44:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca4e7ec700 1 -- 172.21.15.167:0/1314684254 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f39c812180 con 0x55f39c455c00 2024-09-13T23:44:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca4e7ec700 1 -- 172.21.15.167:0/1314684254 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c4f98c0 con 0x55f39c455c00 2024-09-13T23:44:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f39c466820 con 0x55f39c455c00 2024-09-13T23:44:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.737+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f39c4669c0 con 0x55f39c455c00 2024-09-13T23:44:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca4e7ec700 1 -- 172.21.15.167:0/1314684254 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f39c54f080 con 0x55f39c455c00 2024-09-13T23:44:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca4e7ec700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c800c00 0x55f39c73c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca4e7ec700 1 -- 172.21.15.167:0/1314684254 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f39c4bb340 con 0x55f39c455c00 2024-09-13T23:44:29.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f39c801000 0x55f39c73b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c801400 con 0x55f39c801000 2024-09-13T23:44:29.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca54ff9700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c800c00 0x55f39c73c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca55ffb700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f39c801000 0x55f39c73b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca54ff9700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c800c00 0x55f39c73c680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55f39c735920 tx=0x55f39c559800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca55ffb700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f39c801000 0x55f39c73b600 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 20 realm 2024-09-13T23:44:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c801800 con 0x55f39c801000 2024-09-13T23:44:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c801c00 con 0x55f39c801000 2024-09-13T23:44:29.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f39c801400 0x55f39c73bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c801800 con 0x55f39c801400 2024-09-13T23:44:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.741+0000 7fca557fa700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f39c801400 0x55f39c73bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.745+0000 7fca557fa700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f39c801400 0x55f39c73bb80 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.745+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.745+0000 7fca57041f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.745+0000 7fca57041f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.745+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c801c00 con 0x55f39c801400 2024-09-13T23:44:29.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.745+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.745+0000 7fca57041f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:29.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.745+0000 7fca57041f40 20 searching for the correct realm 2024-09-13T23:44:29.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.745+0000 7fca57041f40 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f39c801800 0x55f39c73cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.745+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c801c00 con 0x55f39c801800 2024-09-13T23:44:29.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.745+0000 7fca54ff9700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f39c801800 0x55f39c73cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.745+0000 7fca54ff9700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f39c801800 0x55f39c73cc00 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c801c00 con 0x55f39c801400 2024-09-13T23:44:29.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca557fa700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f39c801c00 0x55f39c73a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801c00 2024-09-13T23:44:29.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca55ffb700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f39c801c00 0x55f39c73a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca55ffb700 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f39c801c00 0x55f39c73a580 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801000 2024-09-13T23:44:29.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801000 2024-09-13T23:44:29.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.749+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801800 2024-09-13T23:44:29.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.753+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.753+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801000 2024-09-13T23:44:29.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.753+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.753+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801400 2024-09-13T23:44:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.753+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.753+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801400 2024-09-13T23:44:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.753+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.753+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801000 2024-09-13T23:44:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.753+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.753+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801c00 2024-09-13T23:44:29.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.753+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.753+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801000 2024-09-13T23:44:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.757+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.757+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801c00 2024-09-13T23:44:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.757+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.757+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801400 2024-09-13T23:44:29.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.757+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.757+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801800 2024-09-13T23:44:29.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.757+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.757+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801000 2024-09-13T23:44:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.757+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.757+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801400 2024-09-13T23:44:29.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.757+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.757+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801800 2024-09-13T23:44:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801000 2024-09-13T23:44:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801c00 2024-09-13T23:44:29.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801000 2024-09-13T23:44:29.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801800 2024-09-13T23:44:29.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801c00 2024-09-13T23:44:29.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801000 2024-09-13T23:44:29.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.761+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801c00 2024-09-13T23:44:29.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.765+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.765+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801800 2024-09-13T23:44:29.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.765+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.765+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801c00 2024-09-13T23:44:29.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.765+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.765+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801400 2024-09-13T23:44:29.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.765+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.765+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801400 2024-09-13T23:44:29.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.765+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.765+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801800 2024-09-13T23:44:29.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.765+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.765+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801400 2024-09-13T23:44:29.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801400 2024-09-13T23:44:29.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:29.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:29.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:29.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:29.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0800 con 0x55f39c801000 2024-09-13T23:44:29.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0c00 con 0x55f39c801400 2024-09-13T23:44:29.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:29.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1000 con 0x55f39c801400 2024-09-13T23:44:29.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:29.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 zone default found 2024-09-13T23:44:29.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 4 Realm: () 2024-09-13T23:44:29.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:29.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:29.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:29.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 zonegroup default 2024-09-13T23:44:29.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.777 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.769+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1400 con 0x55f39c801c00 2024-09-13T23:44:29.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca57041f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca57041f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1800 con 0x55f39c801800 2024-09-13T23:44:29.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca57041f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca57041f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1c00 con 0x55f39c801800 2024-09-13T23:44:29.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca57041f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca57041f40 20 started sync module instance, tier type = 2024-09-13T23:44:29.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca57041f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:29.779 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.773+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1800 con 0x55f39c801400 2024-09-13T23:44:29.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.777+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.777+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94504790203136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1c00 con 0x55f39c801400 2024-09-13T23:44:29.782 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.777+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0400 con 0x55f39c801000 2024-09-13T23:44:29.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.781+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.781+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94504790203136] v35'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.781+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94504790204288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801000 2024-09-13T23:44:29.785 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.781+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0c00 con 0x55f39c801400 2024-09-13T23:44:29.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.781+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94504790204288] v35'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.781+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.781+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94504790205440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1000 con 0x55f39c801400 2024-09-13T23:44:29.788 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.781+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0800 con 0x55f39c801c00 2024-09-13T23:44:29.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.785+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94504790205440] v35'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.785+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.785+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94504790206592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1400 con 0x55f39c801c00 2024-09-13T23:44:29.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.785+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0400 con 0x55f39c801400 2024-09-13T23:44:29.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.789+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94504790206592] v35'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.789+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.789+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94504790207744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0000 con 0x55f39c801400 2024-09-13T23:44:29.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.789+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1800 con 0x55f39c801800 2024-09-13T23:44:29.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.789+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94504790207744] v35'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.789+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.789+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94504790208896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1c00 con 0x55f39c801800 2024-09-13T23:44:29.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.789+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0c00 con 0x55f39c801800 2024-09-13T23:44:29.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94504790208896] v35'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94504790210048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1000 con 0x55f39c801800 2024-09-13T23:44:29.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0400 con 0x55f39c801c00 2024-09-13T23:44:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'207 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94504790211200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1800 con 0x55f39c801c00 2024-09-13T23:44:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca57041f40 20 add_watcher() i=0 2024-09-13T23:44:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca57041f40 20 add_watcher() i=1 2024-09-13T23:44:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca57041f40 20 add_watcher() i=2 2024-09-13T23:44:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca57041f40 20 add_watcher() i=3 2024-09-13T23:44:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca57041f40 20 add_watcher() i=4 2024-09-13T23:44:29.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca57041f40 20 add_watcher() i=5 2024-09-13T23:44:29.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94504790210048] v35'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.793+0000 7fca57041f40 20 add_watcher() i=6 2024-09-13T23:44:29.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.797+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94504790211200] v35'208 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.797+0000 7fca57041f40 20 add_watcher() i=7 2024-09-13T23:44:29.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.797+0000 7fca57041f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:29.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.797+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1c00 con 0x55f39c801000 2024-09-13T23:44:29.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.797+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.797+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94504791900160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c0c00 con 0x55f39c801000 2024-09-13T23:44:29.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94504791900160] v35'208 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca3c7c8700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:29.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca57041f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:29.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca57041f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:29.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca57041f40 1 Processor -- start 2024-09-13T23:44:29.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca57041f40 1 -- start start 2024-09-13T23:44:29.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca57041f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 0x55f39c73d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca57041f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39c47a780 con 0x55f39c8c1400 2024-09-13T23:44:29.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca54ff9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 0x55f39c73d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca54ff9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 0x55f39c73d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42758/0 (socket says 172.21.15.167:42758) 2024-09-13T23:44:29.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca54ff9700 1 -- 172.21.15.167:0/3757951194 learned_addr learned my addr 172.21.15.167:0/3757951194 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca54ff9700 1 -- 172.21.15.167:0/3757951194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f39c466b60 con 0x55f39c8c1400 2024-09-13T23:44:29.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca54ff9700 1 --2- 172.21.15.167:0/3757951194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 0x55f39c73d180 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55f39c8653b0 tx=0x55f39c559860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d561031252a72f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca3bfc7700 1 -- 172.21.15.167:0/3757951194 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c4f9760 con 0x55f39c8c1400 2024-09-13T23:44:29.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca3bfc7700 1 -- 172.21.15.167:0/3757951194 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f39c812600 con 0x55f39c8c1400 2024-09-13T23:44:29.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca3bfc7700 1 -- 172.21.15.167:0/3757951194 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c8e2000 con 0x55f39c8c1400 2024-09-13T23:44:29.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca57041f40 1 -- 172.21.15.167:0/3757951194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 msgr2=0x55f39c73d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca57041f40 1 --2- 172.21.15.167:0/3757951194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 0x55f39c73d180 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x55f39c8653b0 tx=0x55f39c559860 comp rx=0 tx=0).stop 2024-09-13T23:44:29.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca57041f40 1 -- 172.21.15.167:0/3757951194 shutdown_connections 2024-09-13T23:44:29.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca57041f40 1 --2- 172.21.15.167:0/3757951194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 0x55f39c73d180 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca57041f40 1 -- 172.21.15.167:0/3757951194 >> 172.21.15.167:0/3757951194 conn(0x55f39c8c0800 msgr2=0x55f39c556000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.801+0000 7fca57041f40 1 -- 172.21.15.167:0/3757951194 shutdown_connections 2024-09-13T23:44:29.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca57041f40 1 -- 172.21.15.167:0/3757951194 wait complete. 2024-09-13T23:44:29.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca57041f40 1 Processor -- start 2024-09-13T23:44:29.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca57041f40 1 -- start start 2024-09-13T23:44:29.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca57041f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 0x55f39c73d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca57041f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f39c4e2000 con 0x55f39c8c1400 2024-09-13T23:44:29.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca54ff9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 0x55f39c73d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca54ff9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 0x55f39c73d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42766/0 (socket says 172.21.15.167:42766) 2024-09-13T23:44:29.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca54ff9700 1 -- 172.21.15.167:0/3154146357 learned_addr learned my addr 172.21.15.167:0/3154146357 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca54ff9700 1 -- 172.21.15.167:0/3154146357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f39c466d00 con 0x55f39c8c1400 2024-09-13T23:44:29.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca54ff9700 1 --2- 172.21.15.167:0/3154146357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 0x55f39c73d180 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x55f39c8d81e0 tx=0x55f39c8a4420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca3a7c4700 1 -- 172.21.15.167:0/3154146357 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c8e2160 con 0x55f39c8c1400 2024-09-13T23:44:29.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca3a7c4700 1 -- 172.21.15.167:0/3154146357 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f39c812300 con 0x55f39c8c1400 2024-09-13T23:44:29.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca57041f40 1 -- 172.21.15.167:0/3154146357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f39c466340 con 0x55f39c8c1400 2024-09-13T23:44:29.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca3a7c4700 1 -- 172.21.15.167:0/3154146357 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f39c8e22c0 con 0x55f39c8c1400 2024-09-13T23:44:29.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca57041f40 1 -- 172.21.15.167:0/3154146357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f39c467d40 con 0x55f39c8c1400 2024-09-13T23:44:29.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.805+0000 7fca3a7c4700 1 -- 172.21.15.167:0/3154146357 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f39c54fb80 con 0x55f39c8c1400 2024-09-13T23:44:29.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca3a7c4700 1 --2- 172.21.15.167:0/3154146357 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c958000 0x55f39c95e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca3a7c4700 1 -- 172.21.15.167:0/3154146357 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f39c4bb500 con 0x55f39c8c1400 2024-09-13T23:44:29.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca57041f40 5 note: GC not initialized 2024-09-13T23:44:29.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca55ffb700 1 --2- 172.21.15.167:0/3154146357 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c958000 0x55f39c95e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca55ffb700 1 --2- 172.21.15.167:0/3154146357 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c958000 0x55f39c95e000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55f39c865710 tx=0x55f39c7b57a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca37fbf700 20 reqs_thread_entry: start 2024-09-13T23:44:29.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca57041f40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:29.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca57041f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:29.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca35fbb700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:45:00 2024 2024-09-13T23:44:29.815 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:29.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca35799700 20 reqs_thread_entry: start 2024-09-13T23:44:29.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca57041f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c8c1c00 con 0x55f39c801000 2024-09-13T23:44:29.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca57041f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:44:29.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.809+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c958400 con 0x55f39c801800 2024-09-13T23:44:29.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.813+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) v8 ==== 152+0+1268 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.813+0000 7fca3c7c8700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:29.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.813+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94504791900160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c958800 con 0x55f39c801000 2024-09-13T23:44:29.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.813+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94504791900160] v35'209 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.820 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.813+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94504790203136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c958c00 con 0x55f39c801400 2024-09-13T23:44:29.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.817+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94504790203136] v35'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.817+0000 7fca57041f40 20 remove_watcher() i=0 2024-09-13T23:44:29.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.817+0000 7fca57041f40 2 removed watcher, disabling cache 2024-09-13T23:44:29.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.817+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94504790204288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c959000 con 0x55f39c801000 2024-09-13T23:44:29.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.821+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94504790204288] v35'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801000 2024-09-13T23:44:29.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.821+0000 7fca57041f40 20 remove_watcher() i=1 2024-09-13T23:44:29.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.821+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94504790205440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c959400 con 0x55f39c801400 2024-09-13T23:44:29.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.821+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94504790205440] v35'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.821+0000 7fca57041f40 20 remove_watcher() i=2 2024-09-13T23:44:29.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.821+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94504790206592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c959800 con 0x55f39c801c00 2024-09-13T23:44:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.825+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94504790206592] v35'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.825+0000 7fca57041f40 20 remove_watcher() i=3 2024-09-13T23:44:29.830 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.825+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94504790207744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c959c00 con 0x55f39c801400 2024-09-13T23:44:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.829+0000 7fca557fa700 1 -- 172.21.15.167:0/1314684254 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94504790207744] v35'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7200 con 0x55f39c801400 2024-09-13T23:44:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.829+0000 7fca57041f40 20 remove_watcher() i=4 2024-09-13T23:44:29.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.829+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94504790208896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c82a000 con 0x55f39c801800 2024-09-13T23:44:29.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.829+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94504790208896] v35'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.829+0000 7fca57041f40 20 remove_watcher() i=5 2024-09-13T23:44:29.835 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.829+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94504790210048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c82a400 con 0x55f39c801800 2024-09-13T23:44:29.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94504790210048] v35'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b7440 con 0x55f39c801800 2024-09-13T23:44:29.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 20 remove_watcher() i=6 2024-09-13T23:44:29.837 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94504790211200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f39c82a800 con 0x55f39c801c00 2024-09-13T23:44:29.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca55ffb700 1 -- 172.21.15.167:0/1314684254 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94504790211200] v35'209 uv207 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f39c8b6b40 con 0x55f39c801c00 2024-09-13T23:44:29.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 20 remove_watcher() i=7 2024-09-13T23:44:29.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f39c801400 msgr2=0x55f39c73bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f39c801400 0x55f39c73bb80 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.840 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f39c801000 msgr2=0x55f39c73b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f39c801000 0x55f39c73b600 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f39c801c00 msgr2=0x55f39c73a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f39c801c00 0x55f39c73a580 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f39c801800 msgr2=0x55f39c73cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.841 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f39c801800 0x55f39c73cc00 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c800c00 msgr2=0x55f39c73c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c800c00 0x55f39c73c680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55f39c735920 tx=0x55f39c559800 comp rx=0 tx=0).stop 2024-09-13T23:44:29.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 msgr2=0x55f39c73b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 --2- 172.21.15.167:0/1314684254 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455c00 0x55f39c73b080 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x55f39c76c8a0 tx=0x55f39c559f20 comp rx=0 tx=0).stop 2024-09-13T23:44:29.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 reap_dead start 2024-09-13T23:44:29.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca54ff9700 1 -- 172.21.15.167:0/1314684254 reap_dead start 2024-09-13T23:44:29.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 shutdown_connections 2024-09-13T23:44:29.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 >> 172.21.15.167:0/1314684254 conn(0x55f39c455800 msgr2=0x55f39c555800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.833+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 shutdown_connections 2024-09-13T23:44:29.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/1314684254 wait complete. 2024-09-13T23:44:29.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/3154146357 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c958000 msgr2=0x55f39c95e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 --2- 172.21.15.167:0/3154146357 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c958000 0x55f39c95e000 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55f39c865710 tx=0x55f39c7b57a0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/3154146357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 msgr2=0x55f39c73d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 --2- 172.21.15.167:0/3154146357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 0x55f39c73d180 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x55f39c8d81e0 tx=0x55f39c8a4420 comp rx=0 tx=0).stop 2024-09-13T23:44:29.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/3154146357 shutdown_connections 2024-09-13T23:44:29.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 --2- 172.21.15.167:0/3154146357 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c958000 0x55f39c95e000 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 --2- 172.21.15.167:0/3154146357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c8c1400 0x55f39c73d180 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/3154146357 >> 172.21.15.167:0/3154146357 conn(0x55f39c8c0800 msgr2=0x55f39c556000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/3154146357 shutdown_connections 2024-09-13T23:44:29.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/3154146357 wait complete. 2024-09-13T23:44:29.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/503155591 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c800000 msgr2=0x55f39c73c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 --2- 172.21.15.167:0/503155591 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c800000 0x55f39c73c100 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x55f39c76d680 tx=0x55f39c7b4240 comp rx=0 tx=0).stop 2024-09-13T23:44:29.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/503155591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455400 msgr2=0x55f39c73ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 --2- 172.21.15.167:0/503155591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455400 0x55f39c73ab00 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x55f39c5b1f80 tx=0x55f39c559a40 comp rx=0 tx=0).stop 2024-09-13T23:44:29.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/503155591 shutdown_connections 2024-09-13T23:44:29.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 --2- 172.21.15.167:0/503155591 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f39c800000 0x55f39c73c100 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.848 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 --2- 172.21.15.167:0/503155591 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f39c455400 0x55f39c73ab00 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/503155591 >> 172.21.15.167:0/503155591 conn(0x55f39c454c00 msgr2=0x55f39c555000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/503155591 shutdown_connections 2024-09-13T23:44:29.849 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.837+0000 7fca57041f40 1 -- 172.21.15.167:0/503155591 wait complete. 2024-09-13T23:44:29.850 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:44:29.850 INFO:teuthology.orchestra.run.smithi167.stdout: "entries": [ 2024-09-13T23:44:29.850 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.850 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:29.850 INFO:teuthology.orchestra.run.smithi167.stdout: "buckets": [ 2024-09-13T23:44:29.850 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.851 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "", 2024-09-13T23:44:29.851 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.851 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.851 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.851 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [] 2024-09-13T23:44:29.851 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.851 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.851 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo", 2024-09-13T23:44:29.851 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.852 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.852 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.852 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.852 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.852 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:29.852 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 236, 2024-09-13T23:44:29.852 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.852 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 2, 2024-09-13T23:44:29.852 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.852 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.852 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.852 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.853 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.853 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo2", 2024-09-13T23:44:29.853 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.853 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.853 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.853 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.853 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.853 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:29.853 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.854 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.854 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.854 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.854 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.854 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.854 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.854 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.854 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo3", 2024-09-13T23:44:29.854 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.855 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.855 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.855 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.855 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.855 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:29.855 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.855 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.856 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.856 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.856 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.856 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.856 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:29.856 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.856 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket": "myfoo4", 2024-09-13T23:44:29.856 INFO:teuthology.orchestra.run.smithi167.stdout: "time": "2024-09-13T23:00:00.000000Z", 2024-09-13T23:44:29.856 INFO:teuthology.orchestra.run.smithi167.stdout: "epoch": 1726268400, 2024-09-13T23:44:29.856 INFO:teuthology.orchestra.run.smithi167.stdout: "owner": "foo", 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 0, 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 1, 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 1 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.857 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: "summary": [ 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "foo", 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: "categories": [ 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: "category": "delete_bucket", 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 236, 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 5, 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: "total": { 2024-09-13T23:44:29.858 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_sent": 236, 2024-09-13T23:44:29.859 INFO:teuthology.orchestra.run.smithi167.stdout: "bytes_received": 0, 2024-09-13T23:44:29.859 INFO:teuthology.orchestra.run.smithi167.stdout: "ops": 5, 2024-09-13T23:44:29.859 INFO:teuthology.orchestra.run.smithi167.stdout: "successful_ops": 4 2024-09-13T23:44:29.859 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.859 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:29.859 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:44:29.859 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:44:29.859 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}]}, {'bucket': 'myfoo2', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}], 'total': {'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}}]} 2024-09-13T23:44:29.859 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-09-13T23:44:29.860 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-09-13T23:44:29.860 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid bar --display-name Bar --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --max-buckets 4 2024-09-13T23:44:29.948 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:29.948 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:29.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa87702ef40 1 Processor -- start 2024-09-13T23:44:29.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa87702ef40 1 -- start start 2024-09-13T23:44:29.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa87702ef40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56132ad31400 0x56132b016580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa87702ef40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56132adf5500 con 0x56132ad31000 2024-09-13T23:44:29.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa87702ef40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x56132adbea00 con 0x56132ad31400 2024-09-13T23:44:29.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa874fe6700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56132ad31400 0x56132b016580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa8757e7700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x56132ad31000 0x56132ae30800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:36420/0 (socket says 172.21.15.167:36420) 2024-09-13T23:44:29.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa8757e7700 1 -- 172.21.15.167:0/4253201942 learned_addr learned my addr 172.21.15.167:0/4253201942 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:44:29.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa874fe6700 1 -- 172.21.15.167:0/4253201942 >> v1:172.21.15.12:6789/0 conn(0x56132ad31000 legacy=0x56132ae30800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa874fe6700 1 -- 172.21.15.167:0/4253201942 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56132ad43d40 con 0x56132ad31400 2024-09-13T23:44:29.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa874fe6700 1 --2- 172.21.15.167:0/4253201942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31400 0x56132b016580 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x56132b011920 tx=0x56132ae35920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b639217ee8aa73d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa8747e5700 1 -- 172.21.15.167:0/4253201942 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132add54a0 con 0x56132ad31400 2024-09-13T23:44:29.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa8747e5700 1 -- 172.21.15.167:0/4253201942 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56132adf5800 con 0x56132ad31400 2024-09-13T23:44:29.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa8747e5700 1 -- 172.21.15.167:0/4253201942 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132add5600 con 0x56132ad31400 2024-09-13T23:44:29.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa87702ef40 1 -- 172.21.15.167:0/4253201942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31400 msgr2=0x56132b016580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa87702ef40 1 --2- 172.21.15.167:0/4253201942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31400 0x56132b016580 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x56132b011920 tx=0x56132ae35920 comp rx=0 tx=0).stop 2024-09-13T23:44:29.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa87702ef40 1 -- 172.21.15.167:0/4253201942 shutdown_connections 2024-09-13T23:44:29.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa87702ef40 1 --2- 172.21.15.167:0/4253201942 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31400 0x56132b016580 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa87702ef40 1 -- 172.21.15.167:0/4253201942 >> 172.21.15.167:0/4253201942 conn(0x56132ad30c00 msgr2=0x56132ae30000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa87702ef40 1 -- 172.21.15.167:0/4253201942 shutdown_connections 2024-09-13T23:44:29.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.945+0000 7fa87702ef40 1 -- 172.21.15.167:0/4253201942 wait complete. 2024-09-13T23:44:29.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa87702ef40 1 Processor -- start 2024-09-13T23:44:29.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa87702ef40 1 -- start start 2024-09-13T23:44:29.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa87702ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 0x56132b016580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa87702ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132adbe500 con 0x56132ad31000 2024-09-13T23:44:29.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa8757e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 0x56132b016580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa8757e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 0x56132b016580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42774/0 (socket says 172.21.15.167:42774) 2024-09-13T23:44:29.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa8757e7700 1 -- 172.21.15.167:0/3700178197 learned_addr learned my addr 172.21.15.167:0/3700178197 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa8757e7700 1 -- 172.21.15.167:0/3700178197 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56132ad43ba0 con 0x56132ad31000 2024-09-13T23:44:29.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa8757e7700 1 --2- 172.21.15.167:0/3700178197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 0x56132b016580 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x56132bb95ec0 tx=0x56132ae35b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf858efff4f556d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa873fe4700 1 -- 172.21.15.167:0/3700178197 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132add54a0 con 0x56132ad31000 2024-09-13T23:44:29.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa873fe4700 1 -- 172.21.15.167:0/3700178197 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56132adf5b00 con 0x56132ad31000 2024-09-13T23:44:29.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa873fe4700 1 -- 172.21.15.167:0/3700178197 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132add5600 con 0x56132ad31000 2024-09-13T23:44:29.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa87702ef40 1 -- 172.21.15.167:0/3700178197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 msgr2=0x56132b016580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa87702ef40 1 --2- 172.21.15.167:0/3700178197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 0x56132b016580 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x56132bb95ec0 tx=0x56132ae35b60 comp rx=0 tx=0).stop 2024-09-13T23:44:29.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa87702ef40 1 -- 172.21.15.167:0/3700178197 shutdown_connections 2024-09-13T23:44:29.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa87702ef40 1 --2- 172.21.15.167:0/3700178197 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 0x56132b016580 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa87702ef40 1 -- 172.21.15.167:0/3700178197 >> 172.21.15.167:0/3700178197 conn(0x56132ad30c00 msgr2=0x56132ae31000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa87702ef40 1 -- 172.21.15.167:0/3700178197 shutdown_connections 2024-09-13T23:44:29.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.949+0000 7fa87702ef40 1 -- 172.21.15.167:0/3700178197 wait complete. 2024-09-13T23:44:29.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa87702ef40 1 Processor -- start 2024-09-13T23:44:29.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa87702ef40 1 -- start start 2024-09-13T23:44:29.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa87702ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 0x56132b016580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa87702ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132adbe3c0 con 0x56132ad31000 2024-09-13T23:44:29.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa8757e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 0x56132b016580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa8757e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 0x56132b016580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42784/0 (socket says 172.21.15.167:42784) 2024-09-13T23:44:29.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa8757e7700 1 -- 172.21.15.167:0/3507993945 learned_addr learned my addr 172.21.15.167:0/3507993945 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa8757e7700 1 -- 172.21.15.167:0/3507993945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56132ad43a00 con 0x56132ad31000 2024-09-13T23:44:29.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa8757e7700 1 --2- 172.21.15.167:0/3507993945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 0x56132b016580 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x56132bb95f80 tx=0x56132ae35a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa8727e1700 1 -- 172.21.15.167:0/3507993945 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132add58c0 con 0x56132ad31000 2024-09-13T23:44:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa8727e1700 1 -- 172.21.15.167:0/3507993945 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56132adf5c80 con 0x56132ad31000 2024-09-13T23:44:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa8727e1700 1 -- 172.21.15.167:0/3507993945 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132add54a0 con 0x56132ad31000 2024-09-13T23:44:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa87702ef40 1 -- 172.21.15.167:0/3507993945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56132ad43860 con 0x56132ad31000 2024-09-13T23:44:29.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.953+0000 7fa87702ef40 1 -- 172.21.15.167:0/3507993945 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56132ad436c0 con 0x56132ad31000 2024-09-13T23:44:29.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa87702ef40 1 Processor -- start 2024-09-13T23:44:29.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa8727e1700 1 -- 172.21.15.167:0/3507993945 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56132ae2ab00 con 0x56132ad31000 2024-09-13T23:44:29.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa87702ef40 1 -- start start 2024-09-13T23:44:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa8727e1700 1 --2- 172.21.15.167:0/3507993945 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b0d4000 0x56132b017080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa87702ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 0x56132b016b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa87702ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132ad56640 con 0x56132ad31800 2024-09-13T23:44:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa8727e1700 1 -- 172.21.15.167:0/3507993945 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x56132ad97180 con 0x56132ad31000 2024-09-13T23:44:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa875fe8700 1 --2- 172.21.15.167:0/3507993945 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b0d4000 0x56132b017080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa8757e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 0x56132b016b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa8757e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 0x56132b016b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42800/0 (socket says 172.21.15.167:42800) 2024-09-13T23:44:29.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa8757e7700 1 -- 172.21.15.167:0/1676569294 learned_addr learned my addr 172.21.15.167:0/1676569294 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa8757e7700 1 -- 172.21.15.167:0/1676569294 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56132ad42000 con 0x56132ad31800 2024-09-13T23:44:29.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa875fe8700 1 --2- 172.21.15.167:0/3507993945 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b0d4000 0x56132b017080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x56132bba8ae0 tx=0x56132b090de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa8757e7700 1 --2- 172.21.15.167:0/1676569294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 0x56132b016b00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x56132bba9290 tx=0x56132b090240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2184c10c9f521ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa86ffdc700 1 -- 172.21.15.167:0/1676569294 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132add5600 con 0x56132ad31800 2024-09-13T23:44:29.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa86ffdc700 1 -- 172.21.15.167:0/1676569294 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56132adf5c80 con 0x56132ad31800 2024-09-13T23:44:29.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa86ffdc700 1 -- 172.21.15.167:0/1676569294 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132add5a20 con 0x56132ad31800 2024-09-13T23:44:29.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa87702ef40 1 -- 172.21.15.167:0/1676569294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 msgr2=0x56132b016b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:29.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa87702ef40 1 --2- 172.21.15.167:0/1676569294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 0x56132b016b00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x56132bba9290 tx=0x56132b090240 comp rx=0 tx=0).stop 2024-09-13T23:44:29.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa87702ef40 1 -- 172.21.15.167:0/1676569294 shutdown_connections 2024-09-13T23:44:29.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa87702ef40 1 --2- 172.21.15.167:0/1676569294 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 0x56132b016b00 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:29.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa87702ef40 1 -- 172.21.15.167:0/1676569294 >> 172.21.15.167:0/1676569294 conn(0x56132ad31400 msgr2=0x56132ae31800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:29.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.957+0000 7fa87702ef40 1 -- 172.21.15.167:0/1676569294 shutdown_connections 2024-09-13T23:44:29.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa87702ef40 1 -- 172.21.15.167:0/1676569294 wait complete. 2024-09-13T23:44:29.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa87702ef40 1 Processor -- start 2024-09-13T23:44:29.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa87702ef40 1 -- start start 2024-09-13T23:44:29.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa87702ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 0x56132b016b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa87702ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132ad568c0 con 0x56132ad31800 2024-09-13T23:44:29.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa8757e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 0x56132b016b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa8757e7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 0x56132b016b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42812/0 (socket says 172.21.15.167:42812) 2024-09-13T23:44:29.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 learned_addr learned my addr 172.21.15.167:0/662183931 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:29.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56132ad42ea0 con 0x56132ad31800 2024-09-13T23:44:29.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa8757e7700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 0x56132b016b00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x56132bba8360 tx=0x56132b090fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa86e7d9700 1 -- 172.21.15.167:0/662183931 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132add5600 con 0x56132ad31800 2024-09-13T23:44:29.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa86e7d9700 1 -- 172.21.15.167:0/662183931 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56132b0f4000 con 0x56132ad31800 2024-09-13T23:44:29.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa86e7d9700 1 -- 172.21.15.167:0/662183931 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132add5a20 con 0x56132ad31800 2024-09-13T23:44:29.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56132ad42820 con 0x56132ad31800 2024-09-13T23:44:29.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.961+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56132ad429c0 con 0x56132ad31800 2024-09-13T23:44:29.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa86e7d9700 1 -- 172.21.15.167:0/662183931 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56132ae2adc0 con 0x56132ad31800 2024-09-13T23:44:29.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa86e7d9700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b0d5000 0x56132b017600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa86e7d9700 1 -- 172.21.15.167:0/662183931 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x56132ad97340 con 0x56132ad31800 2024-09-13T23:44:29.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa874fe6700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b0d5000 0x56132b017600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56132b0d5400 0x56132b017b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b0d5800 con 0x56132b0d5400 2024-09-13T23:44:29.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa875fe8700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56132b0d5400 0x56132b017b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa874fe6700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b0d5000 0x56132b017600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x56132b011920 tx=0x56132ae35800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa875fe8700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56132b0d5400 0x56132b017b80 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 20 realm 2024-09-13T23:44:29.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b0d5c00 con 0x56132b0d5400 2024-09-13T23:44:29.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:29.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b0d5800 con 0x56132b0d5400 2024-09-13T23:44:29.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56132b0d5c00 0x56132b018100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.965+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e000 con 0x56132b0d5c00 2024-09-13T23:44:29.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa8757e7700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56132b0d5c00 0x56132b018100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa8757e7700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56132b0d5c00 0x56132b018100 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:29.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa87702ef40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:29.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5c00 2024-09-13T23:44:29.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:29.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa87702ef40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:29.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa87702ef40 20 searching for the correct realm 2024-09-13T23:44:29.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa87702ef40 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56132b16e800 0x56132b018680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16ec00 con 0x56132b16e800 2024-09-13T23:44:29.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa874fe6700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56132b16e800 0x56132b018680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa874fe6700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56132b16e800 0x56132b018680 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:29.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.969+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16ec00 con 0x56132b0d5c00 2024-09-13T23:44:29.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:29.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa8757e7700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56132b16e000 0x56132b018c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:29.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e000 2024-09-13T23:44:29.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa875fe8700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56132b16e000 0x56132b018c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:29.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa875fe8700 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56132b16e000 0x56132b018c00 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:29.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:29.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5400 2024-09-13T23:44:29.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5400 2024-09-13T23:44:29.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e800 2024-09-13T23:44:29.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:29.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5400 2024-09-13T23:44:29.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.973+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5c00 2024-09-13T23:44:29.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:29.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5c00 2024-09-13T23:44:29.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:29.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5400 2024-09-13T23:44:29.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e000 2024-09-13T23:44:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5400 2024-09-13T23:44:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e000 2024-09-13T23:44:29.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:29.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.977+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5c00 2024-09-13T23:44:29.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.981+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:29.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.981+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e800 2024-09-13T23:44:29.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.981+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:29.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.981+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5400 2024-09-13T23:44:29.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.981+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.981+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5c00 2024-09-13T23:44:29.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.981+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:29.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.981+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e800 2024-09-13T23:44:29.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.981+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:29.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.981+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5400 2024-09-13T23:44:29.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.981+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.981+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e000 2024-09-13T23:44:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.985+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.985+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5400 2024-09-13T23:44:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.985+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.989 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.985+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e800 2024-09-13T23:44:29.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.985+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:29.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.985+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e000 2024-09-13T23:44:29.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.985+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:29.990 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.985+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5400 2024-09-13T23:44:29.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.985+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.985+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e000 2024-09-13T23:44:29.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.985+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:29.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.985+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e800 2024-09-13T23:44:29.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.989+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:29.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.989+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e000 2024-09-13T23:44:29.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.989+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:29.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.989+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5c00 2024-09-13T23:44:29.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.989+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:29.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.989+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5c00 2024-09-13T23:44:29.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.989+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:29.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.989+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e800 2024-09-13T23:44:29.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.989+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:29.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.989+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5c00 2024-09-13T23:44:29.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:29.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5c00 2024-09-13T23:44:29.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:29.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:29.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:29.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:29.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:29.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16f000 con 0x56132b0d5400 2024-09-13T23:44:29.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:29.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:29.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16f400 con 0x56132b0d5c00 2024-09-13T23:44:29.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:29.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:29.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:29.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16f800 con 0x56132b0d5c00 2024-09-13T23:44:30.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:30.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:30.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 zone default found 2024-09-13T23:44:30.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 4 Realm: () 2024-09-13T23:44:30.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:30.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:30.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:30.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 zonegroup default 2024-09-13T23:44:30.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.993+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16fc00 con 0x56132b16e000 2024-09-13T23:44:30.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:30.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa87702ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:30.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16f000 con 0x56132b16e800 2024-09-13T23:44:30.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:30.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa87702ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:30.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16fc00 con 0x56132b16e800 2024-09-13T23:44:30.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:30.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa87702ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:30.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa87702ef40 20 started sync module instance, tier type = 2024-09-13T23:44:30.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa87702ef40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:30.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:29.997+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b0d5800 con 0x56132b0d5c00 2024-09-13T23:44:30.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.001+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:30.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.001+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94640325843712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16ec00 con 0x56132b0d5c00 2024-09-13T23:44:30.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.001+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b0d5400 2024-09-13T23:44:30.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.005+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:30.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.005+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94640325843712] v35'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:30.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.005+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94640325844864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16f400 con 0x56132b0d5400 2024-09-13T23:44:30.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.005+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16f800 con 0x56132b0d5c00 2024-09-13T23:44:30.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.005+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94640325844864] v35'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:30.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.005+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:30.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.005+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94640325846016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16f000 con 0x56132b0d5c00 2024-09-13T23:44:30.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.005+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16fc00 con 0x56132b16e000 2024-09-13T23:44:30.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.009+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94640325846016] v35'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:30.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.009+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:30.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.009+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94640325847168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b0d4800 con 0x56132b16e000 2024-09-13T23:44:30.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.009+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b0d5800 con 0x56132b0d5c00 2024-09-13T23:44:30.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.013+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94640325847168] v35'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:30.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.013+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:30.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.013+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94640325848320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b0d4800 con 0x56132b0d5c00 2024-09-13T23:44:30.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.013+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e800 2024-09-13T23:44:30.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.013+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:30.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.013+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94640325849472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16f400 con 0x56132b16e800 2024-09-13T23:44:30.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.013+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16fc00 con 0x56132b16e800 2024-09-13T23:44:30.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.013+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94640325848320] v35'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:30.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94640325849472] v35'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:30.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:30.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94640325850624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16e400 con 0x56132b16e800 2024-09-13T23:44:30.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16f400 con 0x56132b16e000 2024-09-13T23:44:30.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'210 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:30.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94640325851776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16fc00 con 0x56132b16e000 2024-09-13T23:44:30.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa87702ef40 20 add_watcher() i=0 2024-09-13T23:44:30.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa87702ef40 20 add_watcher() i=1 2024-09-13T23:44:30.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa87702ef40 20 add_watcher() i=2 2024-09-13T23:44:30.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa87702ef40 20 add_watcher() i=3 2024-09-13T23:44:30.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa87702ef40 20 add_watcher() i=4 2024-09-13T23:44:30.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa87702ef40 20 add_watcher() i=5 2024-09-13T23:44:30.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94640325850624] v35'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:30.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.017+0000 7fa87702ef40 20 add_watcher() i=6 2024-09-13T23:44:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.021+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94640325851776] v35'211 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.021+0000 7fa87702ef40 20 add_watcher() i=7 2024-09-13T23:44:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.021+0000 7fa87702ef40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.021+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b17e000 con 0x56132b0d5400 2024-09-13T23:44:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.021+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:30.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.021+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94640327368704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b17e400 con 0x56132b0d5400 2024-09-13T23:44:30.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94640327368704] v35'210 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:30.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa85c7b5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:30.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa87702ef40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:30.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa87702ef40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:30.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa87702ef40 1 Processor -- start 2024-09-13T23:44:30.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa87702ef40 1 -- start start 2024-09-13T23:44:30.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa87702ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 0x56132b019180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa87702ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132adbe3c0 con 0x56132b17ec00 2024-09-13T23:44:30.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa874fe6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 0x56132b019180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa874fe6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 0x56132b019180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42818/0 (socket says 172.21.15.167:42818) 2024-09-13T23:44:30.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa874fe6700 1 -- 172.21.15.167:0/1675087349 learned_addr learned my addr 172.21.15.167:0/1675087349 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:30.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa874fe6700 1 -- 172.21.15.167:0/1675087349 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56132ad42b60 con 0x56132b17ec00 2024-09-13T23:44:30.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa874fe6700 1 --2- 172.21.15.167:0/1675087349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 0x56132b019180 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x56132b18a090 tx=0x56132ae35860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c04de85b77996b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa85bfb4700 1 -- 172.21.15.167:0/1675087349 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132add5760 con 0x56132b17ec00 2024-09-13T23:44:30.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa85bfb4700 1 -- 172.21.15.167:0/1675087349 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56132b0f4900 con 0x56132b17ec00 2024-09-13T23:44:30.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.025+0000 7fa85bfb4700 1 -- 172.21.15.167:0/1675087349 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132b180160 con 0x56132b17ec00 2024-09-13T23:44:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 -- 172.21.15.167:0/1675087349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 msgr2=0x56132b019180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 --2- 172.21.15.167:0/1675087349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 0x56132b019180 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x56132b18a090 tx=0x56132ae35860 comp rx=0 tx=0).stop 2024-09-13T23:44:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 -- 172.21.15.167:0/1675087349 shutdown_connections 2024-09-13T23:44:30.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 --2- 172.21.15.167:0/1675087349 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 0x56132b019180 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 -- 172.21.15.167:0/1675087349 >> 172.21.15.167:0/1675087349 conn(0x56132b17e800 msgr2=0x56132ae32000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 -- 172.21.15.167:0/1675087349 shutdown_connections 2024-09-13T23:44:30.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 -- 172.21.15.167:0/1675087349 wait complete. 2024-09-13T23:44:30.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 Processor -- start 2024-09-13T23:44:30.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 -- start start 2024-09-13T23:44:30.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 0x56132b019180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56132adbe500 con 0x56132b17ec00 2024-09-13T23:44:30.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa874fe6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 0x56132b019180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa874fe6700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 0x56132b019180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42834/0 (socket says 172.21.15.167:42834) 2024-09-13T23:44:30.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa874fe6700 1 -- 172.21.15.167:0/3247719279 learned_addr learned my addr 172.21.15.167:0/3247719279 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:30.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa874fe6700 1 -- 172.21.15.167:0/3247719279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56132ad42d00 con 0x56132b17ec00 2024-09-13T23:44:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa874fe6700 1 --2- 172.21.15.167:0/3247719279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 0x56132b019180 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x56132b18a2a0 tx=0x56132b14c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa85a7b1700 1 -- 172.21.15.167:0/3247719279 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132b1802c0 con 0x56132b17ec00 2024-09-13T23:44:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa85a7b1700 1 -- 172.21.15.167:0/3247719279 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56132b0f4600 con 0x56132b17ec00 2024-09-13T23:44:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa85a7b1700 1 -- 172.21.15.167:0/3247719279 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56132b180420 con 0x56132b17ec00 2024-09-13T23:44:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 -- 172.21.15.167:0/3247719279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56132ad42340 con 0x56132b17ec00 2024-09-13T23:44:30.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.029+0000 7fa87702ef40 1 -- 172.21.15.167:0/3247719279 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56132ad43d40 con 0x56132b17ec00 2024-09-13T23:44:30.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa85a7b1700 1 -- 172.21.15.167:0/3247719279 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56132ae2bb80 con 0x56132b17ec00 2024-09-13T23:44:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa85a7b1700 1 --2- 172.21.15.167:0/3247719279 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b206000 0x56132b20c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa85a7b1700 1 -- 172.21.15.167:0/3247719279 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x56132ad976c0 con 0x56132b17ec00 2024-09-13T23:44:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa87702ef40 5 note: GC not initialized 2024-09-13T23:44:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa875fe8700 1 --2- 172.21.15.167:0/3247719279 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b206000 0x56132b20c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa875fe8700 1 --2- 172.21.15.167:0/3247719279 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b206000 0x56132b20c000 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x56132b0cf770 tx=0x56132b090d20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa857fac700 20 reqs_thread_entry: start 2024-09-13T23:44:30.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa87702ef40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa87702ef40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa855fa8700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:45:00 2024 2024-09-13T23:44:30.040 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa87702ef40 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-09-13T23:44:30.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b17f800 con 0x56132b16e000 2024-09-13T23:44:30.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.033+0000 7fa855786700 20 reqs_thread_entry: start 2024-09-13T23:44:30.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(62 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:30.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:30.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-09-13T23:44:30.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-09-13T23:44:30.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-09-13T23:44:30.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b17fc00 con 0x56132b0d5400 2024-09-13T23:44:30.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(63 NX5QOQKC6BH2IDN8HC7A [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:30.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:30.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2024-09-13T23:44:30.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-09-13T23:44:30.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-09-13T23:44:30.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-09-13T23:44:30.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:64 8.e 8:718f6a54:users.uid::bar:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b206400 con 0x56132b16e000 2024-09-13T23:44:30.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(64 bar [delete,create,call,writefull 0~346] v35'3 uv3 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:30.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-09-13T23:44:30.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 10 moving default.rgw.meta+users.uid+bar to cache LRU end 2024-09-13T23:44:30.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-09-13T23:44:30.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.037+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:65 7.9 7:93e5b521:::notify.7:head [notify cookie 94640327371008 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b206800 con 0x56132b16e000 2024-09-13T23:44:30.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== watch-notify(notify (1) cookie 94640325851776 notify 150323855381 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x56132b272000 con 0x56132b16e000 2024-09-13T23:44:30.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa86ffdc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855381 cookie 94640325851776 notifier 4790 bl.length()=522 2024-09-13T23:44:30.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa86ffdc700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-09-13T23:44:30.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(65 notify.7 [notify cookie 94640327371008 out=8b] v0'0 uv210 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:30.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa86ffdc700 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar to cache LRU end 2024-09-13T23:44:30.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa86ffdc700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b278000 con 0x56132b16e000 2024-09-13T23:44:30.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== watch-notify(notify_complete (2) cookie 94640327371008 notify 150323855381 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56132b2721a0 con 0x56132b16e000 2024-09-13T23:44:30.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 18 ==== osd_op_reply(66 notify.7 [notify-ack] v0'0 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:30.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b206c00 con 0x56132b0d5400 2024-09-13T23:44:30.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(67 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v35'1 uv1 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:30.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa87702ef40 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-09-13T23:44:30.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa87702ef40 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-09-13T23:44:30.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa87702ef40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-09-13T23:44:30.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.041+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 94640327371008 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b207000 con 0x56132b16e800 2024-09-13T23:44:30.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.045+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== watch-notify(notify (1) cookie 94640325849472 notify 150323855374 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x56132b1d29c0 con 0x56132b16e800 2024-09-13T23:44:30.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.045+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(68 notify.5 [notify cookie 94640327371008 out=8b] v0'0 uv210 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:30.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.045+0000 7fa86ffdc700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855374 cookie 94640325849472 notifier 4790 bl.length()=177 2024-09-13T23:44:30.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.045+0000 7fa86ffdc700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-09-13T23:44:30.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.045+0000 7fa86ffdc700 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-09-13T23:44:30.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.045+0000 7fa86ffdc700 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b278400 con 0x56132b16e800 2024-09-13T23:44:30.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.045+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== watch-notify(notify_complete (2) cookie 94640327371008 notify 150323855374 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56132b1d2b60 con 0x56132b16e800 2024-09-13T23:44:30.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.045+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:30.050 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:44:30.050 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "bar", 2024-09-13T23:44:30.050 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Bar", 2024-09-13T23:44:30.050 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "", 2024-09-13T23:44:30.050 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:44:30.050 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:44:30.050 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:44:30.050 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "bar", 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "NX5QOQKC6BH2IDN8HC7A", 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn" 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:44:30.051 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:30.052 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:44:30.053 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:44:30.053 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:44:30.053 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:44:30.053 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:44:30.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.045+0000 7fa85c7b5700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:30.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.045+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94640327368704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b207400 con 0x56132b0d5400 2024-09-13T23:44:30.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.049+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94640327368704] v35'211 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:30.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.049+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94640325843712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b207800 con 0x56132b0d5c00 2024-09-13T23:44:30.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.049+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94640325843712] v35'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:30.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.049+0000 7fa87702ef40 20 remove_watcher() i=0 2024-09-13T23:44:30.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.049+0000 7fa87702ef40 2 removed watcher, disabling cache 2024-09-13T23:44:30.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.049+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94640325844864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b207c00 con 0x56132b0d5400 2024-09-13T23:44:30.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.053+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94640325844864] v35'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b0d5400 2024-09-13T23:44:30.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.053+0000 7fa87702ef40 20 remove_watcher() i=1 2024-09-13T23:44:30.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.053+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94640325846016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b17e000 con 0x56132b0d5c00 2024-09-13T23:44:30.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.057+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94640325846016] v35'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:30.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.057+0000 7fa87702ef40 20 remove_watcher() i=2 2024-09-13T23:44:30.061 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.057+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94640325847168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b17e400 con 0x56132b16e000 2024-09-13T23:44:30.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.057+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 19 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94640325847168] v35'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:30.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.057+0000 7fa87702ef40 20 remove_watcher() i=3 2024-09-13T23:44:30.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.057+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94640325848320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b17f800 con 0x56132b0d5c00 2024-09-13T23:44:30.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.061+0000 7fa8757e7700 1 -- 172.21.15.167:0/662183931 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94640325848320] v35'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169200 con 0x56132b0d5c00 2024-09-13T23:44:30.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.061+0000 7fa87702ef40 20 remove_watcher() i=4 2024-09-13T23:44:30.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.061+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94640325849472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16f400 con 0x56132b16e800 2024-09-13T23:44:30.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.061+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94640325849472] v35'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:30.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.061+0000 7fa87702ef40 20 remove_watcher() i=5 2024-09-13T23:44:30.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.065+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94640325850624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b16fc00 con 0x56132b16e800 2024-09-13T23:44:30.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.065+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94640325850624] v35'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b169440 con 0x56132b16e800 2024-09-13T23:44:30.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.065+0000 7fa87702ef40 20 remove_watcher() i=6 2024-09-13T23:44:30.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.065+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94640325851776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56132b207000 con 0x56132b16e000 2024-09-13T23:44:30.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa875fe8700 1 -- 172.21.15.167:0/662183931 <== osd.2 v2:172.21.15.167:6808/1180369139 20 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94640325851776] v35'212 uv210 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56132b168b40 con 0x56132b16e000 2024-09-13T23:44:30.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 20 remove_watcher() i=7 2024-09-13T23:44:30.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56132b0d5c00 msgr2=0x56132b018100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56132b0d5c00 0x56132b018100 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56132b0d5400 msgr2=0x56132b017b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56132b0d5400 0x56132b017b80 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56132b16e000 msgr2=0x56132b018c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56132b16e000 0x56132b018c00 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56132b16e800 msgr2=0x56132b018680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56132b16e800 0x56132b018680 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b0d5000 msgr2=0x56132b017600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b0d5000 0x56132b017600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x56132b011920 tx=0x56132ae35800 comp rx=0 tx=0).stop 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 msgr2=0x56132b016b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/662183931 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31800 0x56132b016b00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x56132bba8360 tx=0x56132b090fc0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 reap_dead start 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa874fe6700 1 -- 172.21.15.167:0/662183931 reap_dead start 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 shutdown_connections 2024-09-13T23:44:30.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 >> 172.21.15.167:0/662183931 conn(0x56132ad31400 msgr2=0x56132ae31800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:30.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 shutdown_connections 2024-09-13T23:44:30.076 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/662183931 wait complete. 2024-09-13T23:44:30.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/3247719279 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b206000 msgr2=0x56132b20c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/3247719279 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b206000 0x56132b20c000 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x56132b0cf770 tx=0x56132b090d20 comp rx=0 tx=0).stop 2024-09-13T23:44:30.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/3247719279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 msgr2=0x56132b019180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/3247719279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 0x56132b019180 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x56132b18a2a0 tx=0x56132b14c420 comp rx=0 tx=0).stop 2024-09-13T23:44:30.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/3247719279 shutdown_connections 2024-09-13T23:44:30.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/3247719279 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b206000 0x56132b20c000 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/3247719279 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132b17ec00 0x56132b019180 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/3247719279 >> 172.21.15.167:0/3247719279 conn(0x56132b17e800 msgr2=0x56132ae32000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:30.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/3247719279 shutdown_connections 2024-09-13T23:44:30.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/3247719279 wait complete. 2024-09-13T23:44:30.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/3507993945 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b0d4000 msgr2=0x56132b017080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/3507993945 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b0d4000 0x56132b017080 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x56132bba8ae0 tx=0x56132b090de0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/3507993945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 msgr2=0x56132b016580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/3507993945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 0x56132b016580 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x56132bb95f80 tx=0x56132ae35a40 comp rx=0 tx=0).stop 2024-09-13T23:44:30.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/3507993945 shutdown_connections 2024-09-13T23:44:30.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/3507993945 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56132b0d4000 0x56132b017080 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 --2- 172.21.15.167:0/3507993945 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56132ad31000 0x56132b016580 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.069+0000 7fa87702ef40 1 -- 172.21.15.167:0/3507993945 >> 172.21.15.167:0/3507993945 conn(0x56132ad30c00 msgr2=0x56132ae31000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:30.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.073+0000 7fa87702ef40 1 -- 172.21.15.167:0/3507993945 shutdown_connections 2024-09-13T23:44:30.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.073+0000 7fa87702ef40 1 -- 172.21.15.167:0/3507993945 wait complete. 2024-09-13T23:44:30.081 DEBUG:tasks.util.rgw: json result: {'user_id': 'bar', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bar', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:44:30.112 INFO:tasks.radosgw_admin:RL: create_bucket myfoo6 bar bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:44:30.121 INFO:tasks.radosgw_admin:RL: put_obj myfoo6 bar bytes_out=8 bytes_in=0 failed=False 2024-09-13T23:44:30.121 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-09-13T23:44:30.121 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-09-13T23:44:30.121 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bar --new-uid bud 2024-09-13T23:44:30.161 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:30.161 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190e2acf40 1 Processor -- start 2024-09-13T23:44:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190e2acf40 1 -- start start 2024-09-13T23:44:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190e2acf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55e330a8b000 0x55e330d70580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.162 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190e2acf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55e330b4f500 con 0x55e330a8b400 2024-09-13T23:44:30.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190e2acf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55e330b18a00 con 0x55e330a8b000 2024-09-13T23:44:30.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190ca65700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55e330a8b000 0x55e330d70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190ca65700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55e330a8b000 0x55e330d70580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42844/0 (socket says 172.21.15.167:42844) 2024-09-13T23:44:30.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190ca65700 1 -- 172.21.15.167:0/1183502016 learned_addr learned my addr 172.21.15.167:0/1183502016 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:30.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190c264700 1 --1- 172.21.15.167:0/1183502016 >> v1:172.21.15.12:6789/0 conn(0x55e330a8b400 0x55e330b8a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:36434/0 (socket says 172.21.15.167:36434) 2024-09-13T23:44:30.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190ca65700 1 -- 172.21.15.167:0/1183502016 >> v1:172.21.15.12:6789/0 conn(0x55e330a8b400 legacy=0x55e330b8a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190ca65700 1 -- 172.21.15.167:0/1183502016 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e330a9dd40 con 0x55e330a8b000 2024-09-13T23:44:30.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190ca65700 1 --2- 172.21.15.167:0/1183502016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x55e330d6ba70 tx=0x55e330b8f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492a3f0ec47bf756 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190ba63700 1 -- 172.21.15.167:0/1183502016 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330b2f4a0 con 0x55e330a8b000 2024-09-13T23:44:30.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190ba63700 1 -- 172.21.15.167:0/1183502016 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e330b4f800 con 0x55e330a8b000 2024-09-13T23:44:30.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.157+0000 7f190ba63700 1 -- 172.21.15.167:0/1183502016 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330b2f600 con 0x55e330a8b000 2024-09-13T23:44:30.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 -- 172.21.15.167:0/1183502016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 msgr2=0x55e330d70580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1183502016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x55e330d6ba70 tx=0x55e330b8f920 comp rx=0 tx=0).stop 2024-09-13T23:44:30.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 -- 172.21.15.167:0/1183502016 shutdown_connections 2024-09-13T23:44:30.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1183502016 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 -- 172.21.15.167:0/1183502016 >> 172.21.15.167:0/1183502016 conn(0x55e330a8ac00 msgr2=0x55e330b8a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:30.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 -- 172.21.15.167:0/1183502016 shutdown_connections 2024-09-13T23:44:30.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 -- 172.21.15.167:0/1183502016 wait complete. 2024-09-13T23:44:30.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 Processor -- start 2024-09-13T23:44:30.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 -- start start 2024-09-13T23:44:30.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.167 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e330b18500 con 0x55e330a8b000 2024-09-13T23:44:30.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190ca65700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190ca65700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42860/0 (socket says 172.21.15.167:42860) 2024-09-13T23:44:30.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190ca65700 1 -- 172.21.15.167:0/3450878932 learned_addr learned my addr 172.21.15.167:0/3450878932 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:30.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190ca65700 1 -- 172.21.15.167:0/3450878932 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e330a9dba0 con 0x55e330a8b000 2024-09-13T23:44:30.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190ca65700 1 --2- 172.21.15.167:0/3450878932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x55e330d6ac30 tx=0x55e330b8f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8ff48797bf06af server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190b262700 1 -- 172.21.15.167:0/3450878932 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330b2f760 con 0x55e330a8b000 2024-09-13T23:44:30.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190b262700 1 -- 172.21.15.167:0/3450878932 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e330b4fb00 con 0x55e330a8b000 2024-09-13T23:44:30.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190b262700 1 -- 172.21.15.167:0/3450878932 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330b2f4a0 con 0x55e330a8b000 2024-09-13T23:44:30.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 -- 172.21.15.167:0/3450878932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 msgr2=0x55e330d70580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 --2- 172.21.15.167:0/3450878932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x55e330d6ac30 tx=0x55e330b8f800 comp rx=0 tx=0).stop 2024-09-13T23:44:30.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 -- 172.21.15.167:0/3450878932 shutdown_connections 2024-09-13T23:44:30.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 --2- 172.21.15.167:0/3450878932 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.169 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 -- 172.21.15.167:0/3450878932 >> 172.21.15.167:0/3450878932 conn(0x55e330a8ac00 msgr2=0x55e330b8b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 -- 172.21.15.167:0/3450878932 shutdown_connections 2024-09-13T23:44:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.161+0000 7f190e2acf40 1 -- 172.21.15.167:0/3450878932 wait complete. 2024-09-13T23:44:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190e2acf40 1 Processor -- start 2024-09-13T23:44:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190e2acf40 1 -- start start 2024-09-13T23:44:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190e2acf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190e2acf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e330b183c0 con 0x55e330a8b000 2024-09-13T23:44:30.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190ca65700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190ca65700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42870/0 (socket says 172.21.15.167:42870) 2024-09-13T23:44:30.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190ca65700 1 -- 172.21.15.167:0/2154132684 learned_addr learned my addr 172.21.15.167:0/2154132684 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:30.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190ca65700 1 -- 172.21.15.167:0/2154132684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e330a9da00 con 0x55e330a8b000 2024-09-13T23:44:30.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190ca65700 1 --2- 172.21.15.167:0/2154132684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x55e330be7f80 tx=0x55e330b8f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f1909a5f700 1 -- 172.21.15.167:0/2154132684 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330b2f600 con 0x55e330a8b000 2024-09-13T23:44:30.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f1909a5f700 1 -- 172.21.15.167:0/2154132684 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e330b4fc80 con 0x55e330a8b000 2024-09-13T23:44:30.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190e2acf40 1 -- 172.21.15.167:0/2154132684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e330a9d860 con 0x55e330a8b000 2024-09-13T23:44:30.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f1909a5f700 1 -- 172.21.15.167:0/2154132684 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330b2f8c0 con 0x55e330a8b000 2024-09-13T23:44:30.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190e2acf40 1 -- 172.21.15.167:0/2154132684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e330a9d6c0 con 0x55e330a8b000 2024-09-13T23:44:30.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f1909a5f700 1 -- 172.21.15.167:0/2154132684 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e330b84b00 con 0x55e330a8b000 2024-09-13T23:44:30.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190e2acf40 1 Processor -- start 2024-09-13T23:44:30.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190e2acf40 1 -- start start 2024-09-13T23:44:30.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f1909a5f700 1 --2- 172.21.15.167:0/2154132684 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330e24000 0x55e330d71080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190e2acf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 0x55e330d70b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f1909a5f700 1 -- 172.21.15.167:0/2154132684 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55e330af1180 con 0x55e330a8b000 2024-09-13T23:44:30.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190e2acf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e330ab0640 con 0x55e330a8b800 2024-09-13T23:44:30.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.165+0000 7f190d266700 1 --2- 172.21.15.167:0/2154132684 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330e24000 0x55e330d71080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190ca65700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 0x55e330d70b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190ca65700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 0x55e330d70b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42884/0 (socket says 172.21.15.167:42884) 2024-09-13T23:44:30.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190ca65700 1 -- 172.21.15.167:0/2324506607 learned_addr learned my addr 172.21.15.167:0/2324506607 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:30.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190d266700 1 --2- 172.21.15.167:0/2154132684 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330e24000 0x55e330d71080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55e330be7e30 tx=0x55e3318feb40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190ca65700 1 -- 172.21.15.167:0/2324506607 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e330a9c000 con 0x55e330a8b800 2024-09-13T23:44:30.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190ca65700 1 --2- 172.21.15.167:0/2324506607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 0x55e330d70b00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x55e330daef90 tx=0x55e3318fec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e5fb7c5c2ecf48 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190725a700 1 -- 172.21.15.167:0/2324506607 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330b2f600 con 0x55e330a8b800 2024-09-13T23:44:30.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190725a700 1 -- 172.21.15.167:0/2324506607 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e330b4fc80 con 0x55e330a8b800 2024-09-13T23:44:30.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190725a700 1 -- 172.21.15.167:0/2324506607 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330b2f8c0 con 0x55e330a8b800 2024-09-13T23:44:30.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190e2acf40 1 -- 172.21.15.167:0/2324506607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 msgr2=0x55e330d70b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190e2acf40 1 --2- 172.21.15.167:0/2324506607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 0x55e330d70b00 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x55e330daef90 tx=0x55e3318fec60 comp rx=0 tx=0).stop 2024-09-13T23:44:30.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190e2acf40 1 -- 172.21.15.167:0/2324506607 shutdown_connections 2024-09-13T23:44:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190e2acf40 1 --2- 172.21.15.167:0/2324506607 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 0x55e330d70b00 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190e2acf40 1 -- 172.21.15.167:0/2324506607 >> 172.21.15.167:0/2324506607 conn(0x55e330a8b400 msgr2=0x55e330b8b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190e2acf40 1 -- 172.21.15.167:0/2324506607 shutdown_connections 2024-09-13T23:44:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190e2acf40 1 -- 172.21.15.167:0/2324506607 wait complete. 2024-09-13T23:44:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190e2acf40 1 Processor -- start 2024-09-13T23:44:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190e2acf40 1 -- start start 2024-09-13T23:44:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190e2acf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 0x55e330d70b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190e2acf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e330ab08c0 con 0x55e330a8b800 2024-09-13T23:44:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190ca65700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 0x55e330d70b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190ca65700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 0x55e330d70b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42898/0 (socket says 172.21.15.167:42898) 2024-09-13T23:44:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 learned_addr learned my addr 172.21.15.167:0/1349380541 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:30.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.169+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e330a9cea0 con 0x55e330a8b800 2024-09-13T23:44:30.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f190ca65700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 0x55e330d70b00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55e330dae630 tx=0x55e3318fee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f1905a57700 1 -- 172.21.15.167:0/1349380541 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330b2f760 con 0x55e330a8b800 2024-09-13T23:44:30.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f1905a57700 1 -- 172.21.15.167:0/1349380541 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e330e50000 con 0x55e330a8b800 2024-09-13T23:44:30.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e330a9c820 con 0x55e330a8b800 2024-09-13T23:44:30.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f1905a57700 1 -- 172.21.15.167:0/1349380541 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330b2f4a0 con 0x55e330a8b800 2024-09-13T23:44:30.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e330a9c9c0 con 0x55e330a8b800 2024-09-13T23:44:30.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f1905a57700 1 -- 172.21.15.167:0/1349380541 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e330b84dc0 con 0x55e330a8b800 2024-09-13T23:44:30.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f1905a57700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330e25000 0x55e330d71600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f1905a57700 1 -- 172.21.15.167:0/1349380541 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55e330af1340 con 0x55e330a8b800 2024-09-13T23:44:30.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f190c264700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330e25000 0x55e330d71600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e330e25400 0x55e330d71b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e25800 con 0x55e330e25400 2024-09-13T23:44:30.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f190d266700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e330e25400 0x55e330d71b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f190c264700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330e25000 0x55e330d71600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55e330d6bf80 tx=0x55e330eb8000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.173+0000 7f190d266700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e330e25400 0x55e330d71b80 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:30.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 20 realm 2024-09-13T23:44:30.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e25c00 con 0x55e330e25400 2024-09-13T23:44:30.182 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e25800 con 0x55e330e25400 2024-09-13T23:44:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e330e25c00 0x55e330d72100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2000 con 0x55e330e25c00 2024-09-13T23:44:30.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190ca65700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e330e25c00 0x55e330d72100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190ca65700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e330e25c00 0x55e330d72100 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:30.184 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.177+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25c00 2024-09-13T23:44:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190e2acf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190e2acf40 20 searching for the correct realm 2024-09-13T23:44:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e330ec2800 0x55e330d72680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2c00 con 0x55e330ec2800 2024-09-13T23:44:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190c264700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e330ec2800 0x55e330d72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190c264700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e330ec2800 0x55e330d72680 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2c00 con 0x55e330e25c00 2024-09-13T23:44:30.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190ca65700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e330ec2000 0x55e330d72c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2000 2024-09-13T23:44:30.187 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190d266700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e330ec2000 0x55e330d72c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.181+0000 7f190d266700 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e330ec2000 0x55e330d72c00 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25400 2024-09-13T23:44:30.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.189 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25400 2024-09-13T23:44:30.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2800 2024-09-13T23:44:30.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25400 2024-09-13T23:44:30.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25c00 2024-09-13T23:44:30.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.191 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25c00 2024-09-13T23:44:30.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.185+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25400 2024-09-13T23:44:30.192 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2000 2024-09-13T23:44:30.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25400 2024-09-13T23:44:30.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2000 2024-09-13T23:44:30.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.194 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25c00 2024-09-13T23:44:30.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2800 2024-09-13T23:44:30.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25400 2024-09-13T23:44:30.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.189+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25c00 2024-09-13T23:44:30.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.193+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.193+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2800 2024-09-13T23:44:30.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.193+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.197 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.193+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25400 2024-09-13T23:44:30.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.193+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.193+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2000 2024-09-13T23:44:30.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.193+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.193+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25400 2024-09-13T23:44:30.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.193+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.199 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.193+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2800 2024-09-13T23:44:30.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.193+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.193+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2000 2024-09-13T23:44:30.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25400 2024-09-13T23:44:30.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2000 2024-09-13T23:44:30.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2800 2024-09-13T23:44:30.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2000 2024-09-13T23:44:30.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.202 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25c00 2024-09-13T23:44:30.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25c00 2024-09-13T23:44:30.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.204 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.197+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2800 2024-09-13T23:44:30.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.205 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25c00 2024-09-13T23:44:30.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25c00 2024-09-13T23:44:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190e2acf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190e2acf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190e2acf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190e2acf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3000 con 0x55e330e25400 2024-09-13T23:44:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190e2acf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:30.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.208 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3400 con 0x55e330e25c00 2024-09-13T23:44:30.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190e2acf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.201+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3800 con 0x55e330e25c00 2024-09-13T23:44:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 20 zone default found 2024-09-13T23:44:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 4 Realm: () 2024-09-13T23:44:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 20 zonegroup default 2024-09-13T23:44:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3c00 con 0x55e330ec2000 2024-09-13T23:44:30.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:30.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3000 con 0x55e330ec2800 2024-09-13T23:44:30.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:30.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3c00 con 0x55e330ec2800 2024-09-13T23:44:30.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:30.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 20 started sync module instance, tier type = 2024-09-13T23:44:30.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:30.212 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.205+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e25800 con 0x55e330e25c00 2024-09-13T23:44:30.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.209+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.209+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94434265299712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2c00 con 0x55e330e25c00 2024-09-13T23:44:30.213 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.209+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330e25400 2024-09-13T23:44:30.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.209+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.209+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94434265300864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3400 con 0x55e330e25400 2024-09-13T23:44:30.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.209+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3800 con 0x55e330e25c00 2024-09-13T23:44:30.216 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.209+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94434265299712] v35'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.213+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94434265300864] v35'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.213+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.213+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94434265302016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3000 con 0x55e330e25c00 2024-09-13T23:44:30.218 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.213+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3c00 con 0x55e330ec2000 2024-09-13T23:44:30.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.217+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94434265302016] v35'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.217+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.221 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.217+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94434265303168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e24800 con 0x55e330ec2000 2024-09-13T23:44:30.222 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.217+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e25800 con 0x55e330e25c00 2024-09-13T23:44:30.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.217+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94434265303168] v35'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.217+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.217+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94434265304320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e24800 con 0x55e330e25c00 2024-09-13T23:44:30.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.217+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2800 2024-09-13T23:44:30.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.221+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.221+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94434265305472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3400 con 0x55e330ec2800 2024-09-13T23:44:30.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.221+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3c00 con 0x55e330ec2800 2024-09-13T23:44:30.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.221+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94434265304320] v35'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.221+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94434265305472] v35'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94434265306624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2400 con 0x55e330ec2800 2024-09-13T23:44:30.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3400 con 0x55e330ec2000 2024-09-13T23:44:30.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94434265306624] v35'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'213 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94434265307776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3c00 con 0x55e330ec2000 2024-09-13T23:44:30.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190e2acf40 20 add_watcher() i=0 2024-09-13T23:44:30.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190e2acf40 20 add_watcher() i=1 2024-09-13T23:44:30.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190e2acf40 20 add_watcher() i=2 2024-09-13T23:44:30.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190e2acf40 20 add_watcher() i=3 2024-09-13T23:44:30.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190e2acf40 20 add_watcher() i=4 2024-09-13T23:44:30.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190e2acf40 20 add_watcher() i=5 2024-09-13T23:44:30.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.225+0000 7f190e2acf40 20 add_watcher() i=6 2024-09-13T23:44:30.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.229+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94434265307776] v35'214 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.229+0000 7f190e2acf40 20 add_watcher() i=7 2024-09-13T23:44:30.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.229+0000 7f190e2acf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:30.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.229+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed0000 con 0x55e330e25400 2024-09-13T23:44:30.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.229+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.229+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94434266791936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed0400 con 0x55e330e25400 2024-09-13T23:44:30.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.229+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94434266791936] v35'212 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.229+0000 7f18f3a33700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:30.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.229+0000 7f190e2acf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:30.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.229+0000 7f190e2acf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:30.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.229+0000 7f190e2acf40 1 Processor -- start 2024-09-13T23:44:30.236 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.229+0000 7f190e2acf40 1 -- start start 2024-09-13T23:44:30.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 0x55e330d73180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e330b183c0 con 0x55e330ed0c00 2024-09-13T23:44:30.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190c264700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 0x55e330d73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190c264700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 0x55e330d73180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42910/0 (socket says 172.21.15.167:42910) 2024-09-13T23:44:30.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190c264700 1 -- 172.21.15.167:0/1863025249 learned_addr learned my addr 172.21.15.167:0/1863025249 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:30.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190c264700 1 -- 172.21.15.167:0/1863025249 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e330a9cb60 con 0x55e330ed0c00 2024-09-13T23:44:30.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190c264700 1 --2- 172.21.15.167:0/1863025249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 0x55e330d73180 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x55e330edc660 tx=0x55e330b8ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94d82ed1a69da5c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f18f3232700 1 -- 172.21.15.167:0/1863025249 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330ed2160 con 0x55e330ed0c00 2024-09-13T23:44:30.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f18f3232700 1 -- 172.21.15.167:0/1863025249 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e330e50a80 con 0x55e330ed0c00 2024-09-13T23:44:30.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f18f3232700 1 -- 172.21.15.167:0/1863025249 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330ed22c0 con 0x55e330ed0c00 2024-09-13T23:44:30.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 -- 172.21.15.167:0/1863025249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 msgr2=0x55e330d73180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1863025249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 0x55e330d73180 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x55e330edc660 tx=0x55e330b8ff20 comp rx=0 tx=0).stop 2024-09-13T23:44:30.239 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 -- 172.21.15.167:0/1863025249 shutdown_connections 2024-09-13T23:44:30.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1863025249 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 0x55e330d73180 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 -- 172.21.15.167:0/1863025249 >> 172.21.15.167:0/1863025249 conn(0x55e330ed0800 msgr2=0x55e330b8c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:30.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 -- 172.21.15.167:0/1863025249 shutdown_connections 2024-09-13T23:44:30.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 -- 172.21.15.167:0/1863025249 wait complete. 2024-09-13T23:44:30.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 Processor -- start 2024-09-13T23:44:30.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 -- start start 2024-09-13T23:44:30.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 0x55e330d73180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.233+0000 7f190e2acf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e330b18500 con 0x55e330ed0c00 2024-09-13T23:44:30.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f190c264700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 0x55e330d73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f190c264700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 0x55e330d73180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42922/0 (socket says 172.21.15.167:42922) 2024-09-13T23:44:30.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f190c264700 1 -- 172.21.15.167:0/2660495480 learned_addr learned my addr 172.21.15.167:0/2660495480 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:30.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f190c264700 1 -- 172.21.15.167:0/2660495480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e330a9cd00 con 0x55e330ed0c00 2024-09-13T23:44:30.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f190c264700 1 --2- 172.21.15.167:0/2660495480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 0x55e330d73180 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x55e330edd320 tx=0x55e330eb80c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f18f1a2f700 1 -- 172.21.15.167:0/2660495480 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330ed2420 con 0x55e330ed0c00 2024-09-13T23:44:30.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f18f1a2f700 1 -- 172.21.15.167:0/2660495480 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e330e50780 con 0x55e330ed0c00 2024-09-13T23:44:30.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f190e2acf40 1 -- 172.21.15.167:0/2660495480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e330a9c340 con 0x55e330ed0c00 2024-09-13T23:44:30.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f18f1a2f700 1 -- 172.21.15.167:0/2660495480 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55e330ed2160 con 0x55e330ed0c00 2024-09-13T23:44:30.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f190e2acf40 1 -- 172.21.15.167:0/2660495480 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e330a9cb60 con 0x55e330ed0c00 2024-09-13T23:44:30.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f18f1a2f700 1 -- 172.21.15.167:0/2660495480 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55e330b85b80 con 0x55e330ed0c00 2024-09-13T23:44:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f18f1a2f700 1 --2- 172.21.15.167:0/2660495480 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330f64000 0x55e330f6a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f18f1a2f700 1 -- 172.21.15.167:0/2660495480 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55e330af16c0 con 0x55e330ed0c00 2024-09-13T23:44:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f190d266700 1 --2- 172.21.15.167:0/2660495480 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330f64000 0x55e330f6a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f190e2acf40 5 note: GC not initialized 2024-09-13T23:44:30.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.237+0000 7f190d266700 1 --2- 172.21.15.167:0/2660495480 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330f64000 0x55e330f6a000 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55e330e65f50 tx=0x55e3318fea80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:30.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f18ef22a700 20 reqs_thread_entry: start 2024-09-13T23:44:30.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f190e2acf40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:30.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f190e2acf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f18eca25700 20 reqs_thread_entry: start 2024-09-13T23:44:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f190e2acf40 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-09-13T23:44:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed1800 con 0x55e330ec2000 2024-09-13T23:44:30.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f18ed226700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:45:00 2024 2024-09-13T23:44:30.247 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:30.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(62 bar [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv3 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f190e2acf40 20 rados_obj.operate() r=0 bl.length=346 2024-09-13T23:44:30.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f190e2acf40 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x13 2024-09-13T23:44:30.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f190e2acf40 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-09-13T23:44:30.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f190e2acf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:44:30.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.241+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.4 8:267425a8:users.uid::bud:head [create,call version.set in=58b,writefull 0~181 in=181b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed1c00 con 0x55e330ec2000 2024-09-13T23:44:30.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(63 bud [create,call,writefull 0~181] v35'1 uv1 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190e2acf40 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-09-13T23:44:30.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190e2acf40 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-09-13T23:44:30.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190e2acf40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-09-13T23:44:30.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 7.3 7:c609908c:::notify.5:head [notify cookie 94434266794240 in=369b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f64400 con 0x55e330ec2800 2024-09-13T23:44:30.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== watch-notify(notify (1) cookie 94434265305472 notify 150323855375 ret 0) v3 ==== 399+0+0 (crc 0 0 0) 0x55e330f2eb60 con 0x55e330ec2800 2024-09-13T23:44:30.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190725a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855375 cookie 94434265305472 notifier 4798 bl.length()=357 2024-09-13T23:44:30.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190725a700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-09-13T23:44:30.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190725a700 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-09-13T23:44:30.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(64 notify.5 [notify cookie 94434266794240 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190725a700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330fce000 con 0x55e330ec2800 2024-09-13T23:44:30.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== watch-notify(notify_complete (2) cookie 94434266794240 notify 150323855375 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e330f2ed00 con 0x55e330ec2800 2024-09-13T23:44:30.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(65 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f64800 con 0x55e330e25400 2024-09-13T23:44:30.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(66 bar.buckets [call out=184b] v0'0 uv3 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190e2acf40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 : miss 2024-09-13T23:44:30.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.245+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 8.1a 8:5c084470:root::.bucket.meta.myfoo6%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f64c00 con 0x55e330ec2800 2024-09-13T23:44:30.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(67 .bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 [call out=48b,stat out=16b,read 0~353 out=353b,getxattrs out=218b] v0'0 uv3 ondisk = 0) v8 ==== 333+0+635 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190e2acf40 20 rados_obj.operate() r=0 bl.length=353 2024-09-13T23:44:30.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190e2acf40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 info.flags=0x17 2024-09-13T23:44:30.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190e2acf40 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 to cache LRU end 2024-09-13T23:44:30.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190e2acf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:44:30.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190e2acf40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:44:30.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190e2acf40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 2024-09-13T23:44:30.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 8.1a 8:5c084470:root::.bucket.meta.myfoo6%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f65400 con 0x55e330ec2800 2024-09-13T23:44:30.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(68 .bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 [delete,create,call,call,writefull 0~353,setxattr (117)] v35'4 uv4 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190e2acf40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 info.flags=0x17 2024-09-13T23:44:30.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190e2acf40 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 to cache LRU end 2024-09-13T23:44:30.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190e2acf40 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-09-13T23:44:30.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190e2acf40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7, ofs0, ns] 2024-09-13T23:44:30.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [notify cookie 94434266794240 in=733b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f65c00 con 0x55e330ec2000 2024-09-13T23:44:30.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== watch-notify(notify (1) cookie 94434265303168 notify 150323855384 ret 0) v3 ==== 763+0+0 (crc 0 0 0) 0x55e330fd16c0 con 0x55e330ec2000 2024-09-13T23:44:30.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190725a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855384 cookie 94434265303168 notifier 4798 bl.length()=721 2024-09-13T23:44:30.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190725a700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 info.flags=0x17 2024-09-13T23:44:30.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190725a700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 to cache LRU end 2024-09-13T23:44:30.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190725a700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-09-13T23:44:30.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(69 notify.3 [notify cookie 94434266794240 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.249+0000 7f190725a700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330fce400 con 0x55e330ec2000 2024-09-13T23:44:30.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== watch-notify(notify_complete (2) cookie 94434266794240 notify 150323855384 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e330fd16c0 con 0x55e330ec2000 2024-09-13T23:44:30.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 18 ==== osd_op_reply(70 notify.3 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190e2acf40 20 handle_bi_update(): bucket=:myfoo6[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7]): orig_sources= new_sources= 2024-09-13T23:44:30.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190e2acf40 20 handle_bi_update(): bucket=:myfoo6[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7]): potential sources added=[] removed=[] 2024-09-13T23:44:30.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190e2acf40 20 handle_bi_update(): bucket=:myfoo6[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7]): orig_dests= new_dests= 2024-09-13T23:44:30.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190e2acf40 20 handle_bi_update(): bucket=:myfoo6[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7]): potential dests added=[] removed=[] 2024-09-13T23:44:30.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:71 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed0000 con 0x55e330e25400 2024-09-13T23:44:30.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(71 bar.buckets [call] v35'4 uv4 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190e2acf40 10 cache get: name=default.rgw.meta+root+myfoo6 : miss 2024-09-13T23:44:30.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190e2acf40 20 rados->read ofs=0 len=0 2024-09-13T23:44:30.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed0400 con 0x55e330e25400 2024-09-13T23:44:30.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(72 myfoo6 [call out=48b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv16 ondisk = 0) v8 ==== 234+0+289 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.253+0000 7f190e2acf40 20 rados_obj.operate() r=0 bl.length=166 2024-09-13T23:44:30.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190e2acf40 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x13 2024-09-13T23:44:30.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190e2acf40 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-09-13T23:44:30.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190e2acf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:44:30.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:73 8.18 8:1944a257:root::myfoo6:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~166 in=166b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3400 con 0x55e330e25400 2024-09-13T23:44:30.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(73 myfoo6 [delete,create,call,call,writefull 0~166] v35'17 uv17 ondisk = 0) v8 ==== 318+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190e2acf40 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x17 2024-09-13T23:44:30.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190e2acf40 10 moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-09-13T23:44:30.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190e2acf40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-09-13T23:44:30.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [notify cookie 94434266794240 in=352b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3c00 con 0x55e330ec2000 2024-09-13T23:44:30.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 19 ==== watch-notify(notify (1) cookie 94434265303168 notify 150323855385 ret 0) v3 ==== 382+0+0 (crc 0 0 0) 0x55e330fd16c0 con 0x55e330ec2000 2024-09-13T23:44:30.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190725a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855385 cookie 94434265303168 notifier 4798 bl.length()=340 2024-09-13T23:44:30.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190725a700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x17 2024-09-13T23:44:30.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190725a700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-09-13T23:44:30.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 20 ==== osd_op_reply(74 notify.3 [notify cookie 94434266794240 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190725a700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:75 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed1c00 con 0x55e330ec2000 2024-09-13T23:44:30.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.257+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 21 ==== watch-notify(notify_complete (2) cookie 94434266794240 notify 150323855385 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e330f2eb60 con 0x55e330ec2000 2024-09-13T23:44:30.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.261+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 22 ==== osd_op_reply(75 notify.3 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.261+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:76 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f64800 con 0x55e330e25c00 2024-09-13T23:44:30.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.261+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(76 bud.buckets [call] v35'3 uv3 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.261+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:77 8.18 8:1944a257:root::myfoo6:head [delete,create,call version.inc in=38b,writefull 0~166 in=166b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f65c00 con 0x55e330e25400 2024-09-13T23:44:30.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(77 myfoo6 [delete,create,call,writefull 0~166] v35'18 uv18 ondisk = 0) v8 ==== 276+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190e2acf40 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-09-13T23:44:30.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190e2acf40 10 moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-09-13T23:44:30.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190e2acf40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-09-13T23:44:30.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.e 7:7759931f:::notify.3:head [notify cookie 94434266794240 in=328b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f64400 con 0x55e330ec2000 2024-09-13T23:44:30.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 23 ==== watch-notify(notify (1) cookie 94434265303168 notify 150323855386 ret 0) v3 ==== 358+0+0 (crc 0 0 0) 0x55e330f2eb60 con 0x55e330ec2000 2024-09-13T23:44:30.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190725a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855386 cookie 94434265303168 notifier 4798 bl.length()=316 2024-09-13T23:44:30.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190725a700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-09-13T23:44:30.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190725a700 10 rgw watcher librados: moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-09-13T23:44:30.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 24 ==== osd_op_reply(78 notify.3 [notify cookie 94434266794240 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190725a700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:79 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed0000 con 0x55e330ec2000 2024-09-13T23:44:30.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 25 ==== watch-notify(notify_complete (2) cookie 94434266794240 notify 150323855386 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e330fd1860 con 0x55e330ec2000 2024-09-13T23:44:30.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 26 ==== osd_op_reply(79 notify.3 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 4.1a 4:5a627cc1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f64c00 con 0x55e330e25c00 2024-09-13T23:44:30.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(80 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.0 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.265+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:81 4.12 4:4bdfb1b9:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f65800 con 0x55e330e25400 2024-09-13T23:44:30.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.269+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(81 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.269+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:82 4.36 4:6f4c2246:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed1800 con 0x55e330e25400 2024-09-13T23:44:30.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.269+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(82 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.2 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.269+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:83 4.c 4:3312ce25:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed1c00 con 0x55e330e25400 2024-09-13T23:44:30.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.269+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(83 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.3 [call out=85b] v0'0 uv11 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.269+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:84 4.10 4:09dc872b:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed0000 con 0x55e330ec2800 2024-09-13T23:44:30.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.269+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(84 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.269+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:85 4.12 4:499b0777:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f64800 con 0x55e330e25400 2024-09-13T23:44:30.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.269+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(85 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.5 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.269+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:86 4.3e 4:7ead4b73:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f64c00 con 0x55e330e25c00 2024-09-13T23:44:30.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.273+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(86 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.6 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.273+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:87 4.22 4:44677d15:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f65c00 con 0x55e330e25400 2024-09-13T23:44:30.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.273+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(87 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.273+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:88 4.5 4:a040ca26:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f64400 con 0x55e330e25c00 2024-09-13T23:44:30.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.273+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(88 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.273+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:89 4.34 4:2f86a938:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2c00 con 0x55e330ec2000 2024-09-13T23:44:30.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.273+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 27 ==== osd_op_reply(89 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9 [call out=377b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+377 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.273+0000 7f190e2acf40 10 cls_bucket_list_unordered: got thirteen 2024-09-13T23:44:30.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.273+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:90 4.31 4:8f1da0e5:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3800 con 0x55e330e25c00 2024-09-13T23:44:30.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(90 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 20 get_obj_state: rctx=0x55e330b1b960 obj=myfoo6:thirteen state=0x55e330fda1e8 s->prefetch_data=0 2024-09-13T23:44:30.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:91 3.27s0 3:e602001c:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3000 con 0x55e330ec2800 2024-09-13T23:44:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(91 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7_thirteen [getxattrs out=854b,stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 238+0+870 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 10 manifest: total_size = 8 2024-09-13T23:44:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 20 get_obj_state: setting s->obj_tag to 9144208c-3c85-4fb6-9007-2693d5a930f0.4222.3199321367150124904 2024-09-13T23:44:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 20 Read xattr rgw_rados: user.rgw.acl 2024-09-13T23:44:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 20 Read xattr rgw_rados: user.rgw.content_type 2024-09-13T23:44:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 20 Read xattr rgw_rados: user.rgw.etag 2024-09-13T23:44:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 20 Read xattr rgw_rados: user.rgw.idtag 2024-09-13T23:44:30.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 20 Read xattr rgw_rados: user.rgw.manifest 2024-09-13T23:44:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 20 Read xattr rgw_rados: user.rgw.pg_ver 2024-09-13T23:44:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 20 Read xattr rgw_rados: user.rgw.source_zone 2024-09-13T23:44:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 20 Read xattr rgw_rados: user.rgw.tail_tag 2024-09-13T23:44:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 20 get_obj_state: rctx=0x55e330b1b960 obj=myfoo6:thirteen state=0x55e330fda1e8 s->prefetch_data=0 2024-09-13T23:44:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 : hit (requested=0x17, cached=0x17) 2024-09-13T23:44:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 2024-09-13T23:44:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 20 bucket index object: default.rgw.buckets.index:.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9 2024-09-13T23:44:30.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.277+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:92 4.34 4:2f86a938:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3400 con 0x55e330ec2000 2024-09-13T23:44:30.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.281+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 28 ==== osd_op_reply(92 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9 [stat,call,call] v35'4 uv4 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.281+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:93 3.27s0 3:e602001c:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7_thirteen:head [cmpxattr user.rgw.idtag (62) op 1 mode 1 in=76b,setxattr user.rgw.acl (117) in=129b,setxattr user.rgw.idtag (33) in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3c00 con 0x55e330ec2800 2024-09-13T23:44:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.285+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(93 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7_thirteen [cmpxattr (62) op 1 mode 1,setxattr (117),setxattr (33)] v35'5 uv5 ondisk = 0) v8 ==== 280+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.285+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:94 4.34 4:2f86a938:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=267b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e24800 con 0x55e330ec2000 2024-09-13T23:44:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:1 objects processed in :myfoo6[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7]). Next marker thirteen 2024-09-13T23:44:30.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.285+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:95 8.4 8:267425a8:users.uid::bud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3000 con 0x55e330ec2000 2024-09-13T23:44:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.285+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 29 ==== osd_op_reply(95 bud [delete,create,call,writefull 0~346] v35'2 uv2 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.285+0000 7f190e2acf40 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-09-13T23:44:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.285+0000 7f190e2acf40 10 moving default.rgw.meta+users.uid+bud to cache LRU end 2024-09-13T23:44:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.285+0000 7f190e2acf40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-09-13T23:44:30.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.285+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 30 ==== osd_op_reply(94 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9 [stat,call,call] v35'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.285+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:96 7.3 7:c609908c:::notify.5:head [notify cookie 94434266794240 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3c00 con 0x55e330ec2800 2024-09-13T23:44:30.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.285+0000 7f190725a700 20 handle_completion(): completion ok for obj=thirteen 2024-09-13T23:44:30.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.289+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== watch-notify(notify (1) cookie 94434265305472 notify 150323855376 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x55e330f2ed00 con 0x55e330ec2800 2024-09-13T23:44:30.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.289+0000 7f1906a59700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855376 cookie 94434265305472 notifier 4798 bl.length()=522 2024-09-13T23:44:30.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.289+0000 7f1906a59700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-09-13T23:44:30.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.289+0000 7f1906a59700 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-09-13T23:44:30.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.289+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== osd_op_reply(96 notify.5 [notify cookie 94434266794240 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.289+0000 7f1906a59700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:97 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed1800 con 0x55e330ec2800 2024-09-13T23:44:30.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.289+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 25 ==== watch-notify(notify_complete (2) cookie 94434266794240 notify 150323855376 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e330f2eea0 con 0x55e330ec2800 2024-09-13T23:44:30.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.289+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 26 ==== osd_op_reply(97 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.289+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:98 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ed0000 con 0x55e330e25400 2024-09-13T23:44:30.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 28 ==== osd_op_reply(98 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v35'2 uv2 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190e2acf40 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-09-13T23:44:30.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190e2acf40 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-09-13T23:44:30.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190e2acf40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-09-13T23:44:30.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:99 7.3 7:c609908c:::notify.5:head [notify cookie 94434266794240 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f65000 con 0x55e330ec2800 2024-09-13T23:44:30.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 27 ==== watch-notify(notify (1) cookie 94434265305472 notify 150323855377 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55e330f2eea0 con 0x55e330ec2800 2024-09-13T23:44:30.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f1906a59700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855377 cookie 94434265305472 notifier 4798 bl.length()=177 2024-09-13T23:44:30.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f1906a59700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-09-13T23:44:30.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f1906a59700 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-09-13T23:44:30.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f1906a59700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f65800 con 0x55e330ec2800 2024-09-13T23:44:30.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 28 ==== osd_op_reply(99 notify.5 [notify cookie 94434266794240 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 29 ==== osd_op_reply(100 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 30 ==== watch-notify(notify_complete (2) cookie 94434266794240 notify 150323855377 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e330f2f040 con 0x55e330ec2800 2024-09-13T23:44:30.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190e2acf40 10 removing user index: bar 2024-09-13T23:44:30.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:101 8.e 8:718f6a54:users.uid::bar:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f65400 con 0x55e330ec2000 2024-09-13T23:44:30.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 31 ==== osd_op_reply(101 bar [delete] v35'4 uv4 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190e2acf40 10 removing default.rgw.meta+users.uid+bar from cache 2024-09-13T23:44:30.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190e2acf40 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-09-13T23:44:30.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.293+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [notify cookie 94434266794240 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e24800 con 0x55e330ec2000 2024-09-13T23:44:30.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.297+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 32 ==== watch-notify(notify (1) cookie 94434265307776 notify 150323855387 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x55e330fd1860 con 0x55e330ec2000 2024-09-13T23:44:30.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.297+0000 7f190725a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855387 cookie 94434265307776 notifier 4798 bl.length()=152 2024-09-13T23:44:30.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.297+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 33 ==== osd_op_reply(102 notify.7 [notify cookie 94434266794240 out=8b] v0'0 uv213 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.297+0000 7f190725a700 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e25800 con 0x55e330ec2000 2024-09-13T23:44:30.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.297+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 34 ==== watch-notify(notify_complete (2) cookie 94434266794240 notify 150323855387 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55e330fd1a00 con 0x55e330ec2000 2024-09-13T23:44:30.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.297+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 35 ==== osd_op_reply(103 notify.7 [notify-ack] v0'0 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.302 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:44:30.302 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "bud", 2024-09-13T23:44:30.302 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Bar", 2024-09-13T23:44:30.302 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "", 2024-09-13T23:44:30.302 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:44:30.302 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:44:30.302 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:44:30.302 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:44:30.302 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:30.302 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "bud", 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "NX5QOQKC6BH2IDN8HC7A", 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn" 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:44:30.303 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:44:30.304 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:44:30.305 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:44:30.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.297+0000 7f18f3a33700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:30.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.297+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:104 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94434266791936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f64800 con 0x55e330e25400 2024-09-13T23:44:30.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.301+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 29 ==== osd_op_reply(104 data_loggenerations_metadata [watch unwatch cookie 94434266791936] v35'213 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.301+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:105 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94434265299712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330f65c00 con 0x55e330e25c00 2024-09-13T23:44:30.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.301+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(105 notify.0 [watch unwatch cookie 94434265299712] v35'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.301+0000 7f190e2acf40 20 remove_watcher() i=0 2024-09-13T23:44:30.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.301+0000 7f190e2acf40 2 removed watcher, disabling cache 2024-09-13T23:44:30.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.301+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:106 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94434265300864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec3400 con 0x55e330e25400 2024-09-13T23:44:30.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.305+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.1 v2:172.21.15.167:6800/3875661733 30 ==== osd_op_reply(106 notify.1 [watch unwatch cookie 94434265300864] v35'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330e25400 2024-09-13T23:44:30.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.305+0000 7f190e2acf40 20 remove_watcher() i=1 2024-09-13T23:44:30.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.305+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:107 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94434265302016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330ec2c00 con 0x55e330e25c00 2024-09-13T23:44:30.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.305+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(107 notify.2 [watch unwatch cookie 94434265302016] v35'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.305+0000 7f190e2acf40 20 remove_watcher() i=2 2024-09-13T23:44:30.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.305+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94434265303168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e62000 con 0x55e330ec2000 2024-09-13T23:44:30.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.309+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 36 ==== osd_op_reply(108 notify.3 [watch unwatch cookie 94434265303168] v35'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.309+0000 7f190e2acf40 20 remove_watcher() i=3 2024-09-13T23:44:30.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.309+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:109 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94434265304320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e62400 con 0x55e330e25c00 2024-09-13T23:44:30.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.313+0000 7f190ca65700 1 -- 172.21.15.167:0/1349380541 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(109 notify.4 [watch unwatch cookie 94434265304320] v35'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5200 con 0x55e330e25c00 2024-09-13T23:44:30.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.313+0000 7f190e2acf40 20 remove_watcher() i=4 2024-09-13T23:44:30.317 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.313+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:110 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94434265305472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e62800 con 0x55e330ec2800 2024-09-13T23:44:30.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.313+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 31 ==== osd_op_reply(110 notify.5 [watch unwatch cookie 94434265305472] v35'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.313+0000 7f190e2acf40 20 remove_watcher() i=5 2024-09-13T23:44:30.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.313+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:111 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94434265306624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e62c00 con 0x55e330ec2800 2024-09-13T23:44:30.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 <== osd.3 v2:172.21.15.167:6816/3688363524 32 ==== osd_op_reply(111 notify.6 [watch unwatch cookie 94434265306624] v35'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb5440 con 0x55e330ec2800 2024-09-13T23:44:30.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 20 remove_watcher() i=6 2024-09-13T23:44:30.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:112 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94434265307776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55e330e63000 con 0x55e330ec2000 2024-09-13T23:44:30.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190d266700 1 -- 172.21.15.167:0/1349380541 <== osd.2 v2:172.21.15.167:6808/1180369139 37 ==== osd_op_reply(112 notify.7 [watch unwatch cookie 94434265307776] v35'215 uv213 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55e330eb4b40 con 0x55e330ec2000 2024-09-13T23:44:30.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 20 remove_watcher() i=7 2024-09-13T23:44:30.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e330e25c00 msgr2=0x55e330d72100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55e330e25c00 0x55e330d72100 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e330e25400 msgr2=0x55e330d71b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55e330e25400 0x55e330d71b80 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e330ec2000 msgr2=0x55e330d72c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55e330ec2000 0x55e330d72c00 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e330ec2800 msgr2=0x55e330d72680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55e330ec2800 0x55e330d72680 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330e25000 msgr2=0x55e330d71600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330e25000 0x55e330d71600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55e330d6bf80 tx=0x55e330eb8000 comp rx=0 tx=0).stop 2024-09-13T23:44:30.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 msgr2=0x55e330d70b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 0x55e330d70b00 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x55e330dae630 tx=0x55e3318fee40 comp rx=0 tx=0).stop 2024-09-13T23:44:30.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190c264700 1 -- 172.21.15.167:0/1349380541 reap_dead start 2024-09-13T23:44:30.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 shutdown_connections 2024-09-13T23:44:30.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 --2- 172.21.15.167:0/1349380541 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b800 0x55e330d70b00 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 >> 172.21.15.167:0/1349380541 conn(0x55e330a8b400 msgr2=0x55e330b8b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:30.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 shutdown_connections 2024-09-13T23:44:30.326 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.317+0000 7f190e2acf40 1 -- 172.21.15.167:0/1349380541 wait complete. 2024-09-13T23:44:30.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 -- 172.21.15.167:0/2660495480 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330f64000 msgr2=0x55e330f6a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 --2- 172.21.15.167:0/2660495480 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330f64000 0x55e330f6a000 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55e330e65f50 tx=0x55e3318fea80 comp rx=0 tx=0).stop 2024-09-13T23:44:30.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 -- 172.21.15.167:0/2660495480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 msgr2=0x55e330d73180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 --2- 172.21.15.167:0/2660495480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 0x55e330d73180 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x55e330edd320 tx=0x55e330eb80c0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 -- 172.21.15.167:0/2660495480 shutdown_connections 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 --2- 172.21.15.167:0/2660495480 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330f64000 0x55e330f6a000 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 --2- 172.21.15.167:0/2660495480 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330ed0c00 0x55e330d73180 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 -- 172.21.15.167:0/2660495480 >> 172.21.15.167:0/2660495480 conn(0x55e330ed0800 msgr2=0x55e330b8c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 -- 172.21.15.167:0/2660495480 shutdown_connections 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 -- 172.21.15.167:0/2660495480 wait complete. 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 -- 172.21.15.167:0/2154132684 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330e24000 msgr2=0x55e330d71080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 --2- 172.21.15.167:0/2154132684 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330e24000 0x55e330d71080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55e330be7e30 tx=0x55e3318feb40 comp rx=0 tx=0).stop 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 -- 172.21.15.167:0/2154132684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 msgr2=0x55e330d70580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 --2- 172.21.15.167:0/2154132684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x55e330be7f80 tx=0x55e330b8f920 comp rx=0 tx=0).stop 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 -- 172.21.15.167:0/2154132684 shutdown_connections 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 --2- 172.21.15.167:0/2154132684 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55e330e24000 0x55e330d71080 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 --2- 172.21.15.167:0/2154132684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55e330a8b000 0x55e330d70580 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:30.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 -- 172.21.15.167:0/2154132684 >> 172.21.15.167:0/2154132684 conn(0x55e330a8ac00 msgr2=0x55e330b8b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:30.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 -- 172.21.15.167:0/2154132684 shutdown_connections 2024-09-13T23:44:30.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:30.321+0000 7f190e2acf40 1 -- 172.21.15.167:0/2154132684 wait complete. 2024-09-13T23:44:30.330 DEBUG:tasks.util.rgw: json result: {'user_id': 'bud', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bud', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:44:35.342 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:44:35.342 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-09-13T23:44:35.342 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-09-13T23:44:35.342 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-09-13T23:44:35.343 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 4 2024-09-13T23:44:35.391 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:35.391 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:35.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.385+0000 7f3a8281af40 1 Processor -- start 2024-09-13T23:44:35.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.385+0000 7f3a8281af40 1 -- start start 2024-09-13T23:44:35.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.385+0000 7f3a8281af40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56144ce95000 0x56144d17a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x56144cf59500 con 0x56144ce95400 2024-09-13T23:44:35.392 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x56144cf22a00 con 0x56144ce95000 2024-09-13T23:44:35.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a80fd3700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x56144ce95000 0x56144d17a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a807d2700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x56144ce95400 0x56144cf94800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:36448/0 (socket says 172.21.15.167:36448) 2024-09-13T23:44:35.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a807d2700 1 -- 172.21.15.167:0/3175995738 learned_addr learned my addr 172.21.15.167:0/3175995738 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:44:35.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a80fd3700 1 -- 172.21.15.167:0/3175995738 >> v1:172.21.15.12:6789/0 conn(0x56144ce95400 legacy=0x56144cf94800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a7ffd1700 1 -- 172.21.15.167:0/3175995738 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466829303 0 0) 0x56144cff0000 con 0x56144ce95400 2024-09-13T23:44:35.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a80fd3700 1 -- 172.21.15.167:0/3175995738 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56144cea7d40 con 0x56144ce95000 2024-09-13T23:44:35.394 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a80fd3700 1 --2- 172.21.15.167:0/3175995738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95000 0x56144d17a580 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x56144d175a70 tx=0x56144cf99920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bf5d1613f38565 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a7ffd1700 1 -- 172.21.15.167:0/3175995738 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144cf394a0 con 0x56144ce95000 2024-09-13T23:44:35.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a7ffd1700 1 -- 172.21.15.167:0/3175995738 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56144cf59800 con 0x56144ce95000 2024-09-13T23:44:35.395 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a7ffd1700 1 -- 172.21.15.167:0/3175995738 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144cf39600 con 0x56144ce95000 2024-09-13T23:44:35.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 -- 172.21.15.167:0/3175995738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95000 msgr2=0x56144d17a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 --2- 172.21.15.167:0/3175995738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95000 0x56144d17a580 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x56144d175a70 tx=0x56144cf99920 comp rx=0 tx=0).stop 2024-09-13T23:44:35.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 -- 172.21.15.167:0/3175995738 shutdown_connections 2024-09-13T23:44:35.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 --2- 172.21.15.167:0/3175995738 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95000 0x56144d17a580 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 -- 172.21.15.167:0/3175995738 >> 172.21.15.167:0/3175995738 conn(0x56144ce94c00 msgr2=0x56144cf94000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 -- 172.21.15.167:0/3175995738 shutdown_connections 2024-09-13T23:44:35.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 -- 172.21.15.167:0/3175995738 wait complete. 2024-09-13T23:44:35.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 Processor -- start 2024-09-13T23:44:35.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 -- start start 2024-09-13T23:44:35.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95000 0x56144d17a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.397 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.389+0000 7f3a8281af40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144cf22500 con 0x56144ce95000 2024-09-13T23:44:35.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a80fd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95000 0x56144d17a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a80fd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95000 0x56144d17a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42944/0 (socket says 172.21.15.167:42944) 2024-09-13T23:44:35.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a80fd3700 1 -- 172.21.15.167:0/3462398798 learned_addr learned my addr 172.21.15.167:0/3462398798 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a80fd3700 1 -- 172.21.15.167:0/3462398798 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56144cea7ba0 con 0x56144ce95000 2024-09-13T23:44:35.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a80fd3700 1 --2- 172.21.15.167:0/3462398798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95000 0x56144d17a580 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x56144dc69e00 tx=0x56144cf99800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1181a3fc69241f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a7f7d0700 1 -- 172.21.15.167:0/3462398798 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144cf39760 con 0x56144ce95000 2024-09-13T23:44:35.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a7f7d0700 1 -- 172.21.15.167:0/3462398798 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56144cf59b00 con 0x56144ce95000 2024-09-13T23:44:35.398 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a7f7d0700 1 -- 172.21.15.167:0/3462398798 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144cf391e0 con 0x56144ce95000 2024-09-13T23:44:35.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a8281af40 1 -- 172.21.15.167:0/3462398798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95000 msgr2=0x56144d17a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a8281af40 1 --2- 172.21.15.167:0/3462398798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95000 0x56144d17a580 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x56144dc69e00 tx=0x56144cf99800 comp rx=0 tx=0).stop 2024-09-13T23:44:35.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a8281af40 1 -- 172.21.15.167:0/3462398798 shutdown_connections 2024-09-13T23:44:35.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a8281af40 1 --2- 172.21.15.167:0/3462398798 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95000 0x56144d17a580 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a8281af40 1 -- 172.21.15.167:0/3462398798 >> 172.21.15.167:0/3462398798 conn(0x56144ce94c00 msgr2=0x56144cf95000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a8281af40 1 -- 172.21.15.167:0/3462398798 shutdown_connections 2024-09-13T23:44:35.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a8281af40 1 -- 172.21.15.167:0/3462398798 wait complete. 2024-09-13T23:44:35.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a8281af40 1 Processor -- start 2024-09-13T23:44:35.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a8281af40 1 -- start start 2024-09-13T23:44:35.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a8281af40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95400 0x56144d17ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a8281af40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144cf223c0 con 0x56144ce95400 2024-09-13T23:44:35.400 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a80fd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95400 0x56144d17ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a80fd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95400 0x56144d17ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42954/0 (socket says 172.21.15.167:42954) 2024-09-13T23:44:35.401 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2236064158 learned_addr learned my addr 172.21.15.167:0/2236064158 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.393+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2236064158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56144cea7a00 con 0x56144ce95400 2024-09-13T23:44:35.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a80fd3700 1 --2- 172.21.15.167:0/2236064158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95400 0x56144d17ab00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x56144dc69f50 tx=0x56144cf99920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a7dfcd700 1 -- 172.21.15.167:0/2236064158 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144cf394a0 con 0x56144ce95400 2024-09-13T23:44:35.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a7dfcd700 1 -- 172.21.15.167:0/2236064158 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56144cf59c80 con 0x56144ce95400 2024-09-13T23:44:35.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a7dfcd700 1 -- 172.21.15.167:0/2236064158 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144cf39600 con 0x56144ce95400 2024-09-13T23:44:35.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a8281af40 1 -- 172.21.15.167:0/2236064158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56144cea7860 con 0x56144ce95400 2024-09-13T23:44:35.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a8281af40 1 -- 172.21.15.167:0/2236064158 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56144cea76c0 con 0x56144ce95400 2024-09-13T23:44:35.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a8281af40 1 Processor -- start 2024-09-13T23:44:35.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a8281af40 1 -- start start 2024-09-13T23:44:35.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a7dfcd700 1 -- 172.21.15.167:0/2236064158 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56144cf8eb00 con 0x56144ce95400 2024-09-13T23:44:35.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a7dfcd700 1 --2- 172.21.15.167:0/2236064158 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144ce95c00 0x56144d17b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a7dfcd700 1 -- 172.21.15.167:0/2236064158 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x56144cefb180 con 0x56144ce95400 2024-09-13T23:44:35.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a8281af40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 0x56144d17b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.403 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a8281af40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144ceba640 con 0x56144d242000 2024-09-13T23:44:35.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a817d4700 1 --2- 172.21.15.167:0/2236064158 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144ce95c00 0x56144d17b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a80fd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 0x56144d17b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a80fd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 0x56144d17b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42956/0 (socket says 172.21.15.167:42956) 2024-09-13T23:44:35.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.397+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2191987576 learned_addr learned my addr 172.21.15.167:0/2191987576 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a817d4700 1 --2- 172.21.15.167:0/2236064158 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144ce95c00 0x56144d17b080 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x56144d1c2300 tx=0x56144d202b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2191987576 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56144cea6000 con 0x56144d242000 2024-09-13T23:44:35.405 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a80fd3700 1 --2- 172.21.15.167:0/2191987576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 0x56144d17b600 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x56144d1c3560 tx=0x56144d202c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed8ab16ea3200c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a7b7c8700 1 -- 172.21.15.167:0/2191987576 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144cf394a0 con 0x56144d242000 2024-09-13T23:44:35.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a7b7c8700 1 -- 172.21.15.167:0/2191987576 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56144cf59c80 con 0x56144d242000 2024-09-13T23:44:35.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a7b7c8700 1 -- 172.21.15.167:0/2191987576 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144cf39600 con 0x56144d242000 2024-09-13T23:44:35.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a8281af40 1 -- 172.21.15.167:0/2191987576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 msgr2=0x56144d17b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2191987576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 0x56144d17b600 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x56144d1c3560 tx=0x56144d202c60 comp rx=0 tx=0).stop 2024-09-13T23:44:35.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a8281af40 1 -- 172.21.15.167:0/2191987576 shutdown_connections 2024-09-13T23:44:35.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2191987576 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 0x56144d17b600 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a8281af40 1 -- 172.21.15.167:0/2191987576 >> 172.21.15.167:0/2191987576 conn(0x56144ce95800 msgr2=0x56144cf95800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a8281af40 1 -- 172.21.15.167:0/2191987576 shutdown_connections 2024-09-13T23:44:35.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a8281af40 1 -- 172.21.15.167:0/2191987576 wait complete. 2024-09-13T23:44:35.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a8281af40 1 Processor -- start 2024-09-13T23:44:35.407 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a8281af40 1 -- start start 2024-09-13T23:44:35.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a8281af40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 0x56144d17b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a8281af40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144ceba8c0 con 0x56144d242000 2024-09-13T23:44:35.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a80fd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 0x56144d17b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a80fd3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 0x56144d17b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42966/0 (socket says 172.21.15.167:42966) 2024-09-13T23:44:35.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 learned_addr learned my addr 172.21.15.167:0/2790292812 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.408 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.401+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56144cea6ea0 con 0x56144d242000 2024-09-13T23:44:35.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a80fd3700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 0x56144d17b600 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x56144d1c21b0 tx=0x56144d202e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a79fc5700 1 -- 172.21.15.167:0/2790292812 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144cf39760 con 0x56144d242000 2024-09-13T23:44:35.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a79fc5700 1 -- 172.21.15.167:0/2790292812 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56144d262000 con 0x56144d242000 2024-09-13T23:44:35.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56144cea6820 con 0x56144d242000 2024-09-13T23:44:35.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a79fc5700 1 -- 172.21.15.167:0/2790292812 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144cf391e0 con 0x56144d242000 2024-09-13T23:44:35.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56144cea69c0 con 0x56144d242000 2024-09-13T23:44:35.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a79fc5700 1 -- 172.21.15.167:0/2790292812 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56144cf8edc0 con 0x56144d242000 2024-09-13T23:44:35.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a79fc5700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144d243800 0x56144d17c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a79fc5700 1 -- 172.21.15.167:0/2790292812 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x56144cefb340 con 0x56144d242000 2024-09-13T23:44:35.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a807d2700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144d243800 0x56144d17c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56144d242400 0x56144d17bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d242800 con 0x56144d242400 2024-09-13T23:44:35.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a817d4700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56144d242400 0x56144d17bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a807d2700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144d243800 0x56144d17c680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x56144dc68030 tx=0x56144d2cc000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.405+0000 7f3a817d4700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56144d242400 0x56144d17bb80 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 20 realm 2024-09-13T23:44:35.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d242c00 con 0x56144d242400 2024-09-13T23:44:35.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:35.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d242800 con 0x56144d242400 2024-09-13T23:44:35.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56144d242c00 0x56144d17c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d243c00 con 0x56144d242c00 2024-09-13T23:44:35.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a80fd3700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56144d242c00 0x56144d17c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a80fd3700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56144d242c00 0x56144d17c100 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:35.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0000 con 0x56144d242c00 2024-09-13T23:44:35.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:35.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 20 searching for the correct realm 2024-09-13T23:44:35.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.409+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56144d2e0400 0x56144d17cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0400 2024-09-13T23:44:35.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a807d2700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56144d2e0400 0x56144d17cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a807d2700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56144d2e0400 0x56144d17cc00 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242c00 2024-09-13T23:44:35.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a80fd3700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56144d2e0000 0x56144d17a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0000 2024-09-13T23:44:35.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a817d4700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56144d2e0000 0x56144d17a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a817d4700 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56144d2e0000 0x56144d17a580 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242400 2024-09-13T23:44:35.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.413+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242400 2024-09-13T23:44:35.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0400 2024-09-13T23:44:35.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242400 2024-09-13T23:44:35.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242c00 2024-09-13T23:44:35.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242c00 2024-09-13T23:44:35.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242400 2024-09-13T23:44:35.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0000 2024-09-13T23:44:35.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.417+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242400 2024-09-13T23:44:35.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.421+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.421+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0000 2024-09-13T23:44:35.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.421+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.421+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242c00 2024-09-13T23:44:35.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.421+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.421+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0400 2024-09-13T23:44:35.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.421+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.421+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242400 2024-09-13T23:44:35.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.421+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.421+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242c00 2024-09-13T23:44:35.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.421+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.421+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0400 2024-09-13T23:44:35.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242400 2024-09-13T23:44:35.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0000 2024-09-13T23:44:35.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242400 2024-09-13T23:44:35.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0400 2024-09-13T23:44:35.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0000 2024-09-13T23:44:35.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242400 2024-09-13T23:44:35.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.425+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0000 2024-09-13T23:44:35.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.429+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.429+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0400 2024-09-13T23:44:35.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.429+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.429+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0000 2024-09-13T23:44:35.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.429+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.429+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242c00 2024-09-13T23:44:35.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.429+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.429+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242c00 2024-09-13T23:44:35.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.429+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.429+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d2e0400 2024-09-13T23:44:35.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.429+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.429+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242c00 2024-09-13T23:44:35.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242c00 2024-09-13T23:44:35.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:35.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:35.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:35.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:35.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0c00 con 0x56144d242400 2024-09-13T23:44:35.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1000 con 0x56144d242c00 2024-09-13T23:44:35.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:35.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1400 con 0x56144d242c00 2024-09-13T23:44:35.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:35.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 zone default found 2024-09-13T23:44:35.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 4 Realm: () 2024-09-13T23:44:35.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:35.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:35.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:35.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 zonegroup default 2024-09-13T23:44:35.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.433+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1800 con 0x56144d2e0000 2024-09-13T23:44:35.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a8281af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1c00 con 0x56144d2e0400 2024-09-13T23:44:35.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a8281af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0c00 con 0x56144d2e0400 2024-09-13T23:44:35.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a8281af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a8281af40 20 started sync module instance, tier type = 2024-09-13T23:44:35.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a8281af40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:35.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.437+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1800 con 0x56144d242c00 2024-09-13T23:44:35.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.441+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.441+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94645192694528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d242800 con 0x56144d242c00 2024-09-13T23:44:35.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.441+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242400 2024-09-13T23:44:35.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.441+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.441+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94645192695680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1000 con 0x56144d242400 2024-09-13T23:44:35.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.441+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1400 con 0x56144d242c00 2024-09-13T23:44:35.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.441+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94645192694528] v35'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.445+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.445+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94645192696832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1800 con 0x56144d242c00 2024-09-13T23:44:35.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.445+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1c00 con 0x56144d2e0000 2024-09-13T23:44:35.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.445+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94645192695680] v35'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.449+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94645192696832] v35'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.449+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.449+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94645192697984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0c00 con 0x56144d2e0000 2024-09-13T23:44:35.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.449+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d243c00 con 0x56144d242c00 2024-09-13T23:44:35.456 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.449+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94645192697984] v35'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.453+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.453+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94645192699136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0800 con 0x56144d242c00 2024-09-13T23:44:35.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.453+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1000 con 0x56144d2e0400 2024-09-13T23:44:35.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.453+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.453+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94645192700288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1c00 con 0x56144d2e0400 2024-09-13T23:44:35.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.453+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0c00 con 0x56144d2e0400 2024-09-13T23:44:35.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.453+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94645192699136] v35'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94645192700288] v35'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94645192701440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1000 con 0x56144d2e0400 2024-09-13T23:44:35.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1c00 con 0x56144d2e0000 2024-09-13T23:44:35.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94645192701440] v35'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'216 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94645192702592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0c00 con 0x56144d2e0000 2024-09-13T23:44:35.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a8281af40 20 add_watcher() i=0 2024-09-13T23:44:35.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a8281af40 20 add_watcher() i=1 2024-09-13T23:44:35.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a8281af40 20 add_watcher() i=2 2024-09-13T23:44:35.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a8281af40 20 add_watcher() i=3 2024-09-13T23:44:35.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a8281af40 20 add_watcher() i=4 2024-09-13T23:44:35.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a8281af40 20 add_watcher() i=5 2024-09-13T23:44:35.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.457+0000 7f3a8281af40 20 add_watcher() i=6 2024-09-13T23:44:35.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.461+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94645192702592] v35'217 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.461+0000 7f3a8281af40 20 add_watcher() i=7 2024-09-13T23:44:35.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.461+0000 7f3a8281af40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:35.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.461+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2ec000 con 0x56144d242400 2024-09-13T23:44:35.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.461+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.461+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94645194252288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2ec400 con 0x56144d242400 2024-09-13T23:44:35.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.461+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94645194252288] v35'214 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a67fa1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:35.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a8281af40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:35.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a8281af40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:35.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a8281af40 1 Processor -- start 2024-09-13T23:44:35.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a8281af40 1 -- start start 2024-09-13T23:44:35.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a8281af40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 0x56144d17d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a8281af40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144cf223c0 con 0x56144d2ecc00 2024-09-13T23:44:35.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a807d2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 0x56144d17d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a807d2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 0x56144d17d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42970/0 (socket says 172.21.15.167:42970) 2024-09-13T23:44:35.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a807d2700 1 -- 172.21.15.167:0/3594226332 learned_addr learned my addr 172.21.15.167:0/3594226332 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a807d2700 1 -- 172.21.15.167:0/3594226332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56144cea6b60 con 0x56144d2ecc00 2024-09-13T23:44:35.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a807d2700 1 --2- 172.21.15.167:0/3594226332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 0x56144d17d180 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x56144d2f6030 tx=0x56144d203f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47aed23f2e4ae50 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a677a0700 1 -- 172.21.15.167:0/3594226332 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144d2fe000 con 0x56144d2ecc00 2024-09-13T23:44:35.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a677a0700 1 -- 172.21.15.167:0/3594226332 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56144d262a80 con 0x56144d2ecc00 2024-09-13T23:44:35.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a677a0700 1 -- 172.21.15.167:0/3594226332 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144d2fe160 con 0x56144d2ecc00 2024-09-13T23:44:35.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a8281af40 1 -- 172.21.15.167:0/3594226332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 msgr2=0x56144d17d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a8281af40 1 --2- 172.21.15.167:0/3594226332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 0x56144d17d180 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x56144d2f6030 tx=0x56144d203f20 comp rx=0 tx=0).stop 2024-09-13T23:44:35.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a8281af40 1 -- 172.21.15.167:0/3594226332 shutdown_connections 2024-09-13T23:44:35.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a8281af40 1 --2- 172.21.15.167:0/3594226332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 0x56144d17d180 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.465+0000 7f3a8281af40 1 -- 172.21.15.167:0/3594226332 >> 172.21.15.167:0/3594226332 conn(0x56144d2ec800 msgr2=0x56144cf96000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a8281af40 1 -- 172.21.15.167:0/3594226332 shutdown_connections 2024-09-13T23:44:35.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a8281af40 1 -- 172.21.15.167:0/3594226332 wait complete. 2024-09-13T23:44:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a8281af40 1 Processor -- start 2024-09-13T23:44:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a8281af40 1 -- start start 2024-09-13T23:44:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a8281af40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 0x56144d17d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a8281af40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x56144cf22500 con 0x56144d2ecc00 2024-09-13T23:44:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a807d2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 0x56144d17d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a807d2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 0x56144d17d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42974/0 (socket says 172.21.15.167:42974) 2024-09-13T23:44:35.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a807d2700 1 -- 172.21.15.167:0/1166185227 learned_addr learned my addr 172.21.15.167:0/1166185227 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a807d2700 1 -- 172.21.15.167:0/1166185227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56144cea6d00 con 0x56144d2ecc00 2024-09-13T23:44:35.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a807d2700 1 --2- 172.21.15.167:0/1166185227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 0x56144d17d180 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x56144d2f6240 tx=0x56144d2cc0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a65f9d700 1 -- 172.21.15.167:0/1166185227 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144d2fe2c0 con 0x56144d2ecc00 2024-09-13T23:44:35.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a65f9d700 1 -- 172.21.15.167:0/1166185227 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56144d262780 con 0x56144d2ecc00 2024-09-13T23:44:35.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a8281af40 1 -- 172.21.15.167:0/1166185227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56144cea6340 con 0x56144d2ecc00 2024-09-13T23:44:35.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a65f9d700 1 -- 172.21.15.167:0/1166185227 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x56144d2fe000 con 0x56144d2ecc00 2024-09-13T23:44:35.475 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.469+0000 7f3a8281af40 1 -- 172.21.15.167:0/1166185227 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56144cea6b60 con 0x56144d2ecc00 2024-09-13T23:44:35.476 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a65f9d700 1 -- 172.21.15.167:0/1166185227 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x56144cf8f8c0 con 0x56144d2ecc00 2024-09-13T23:44:35.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a65f9d700 1 --2- 172.21.15.167:0/1166185227 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144d37a000 0x56144d380000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a65f9d700 1 -- 172.21.15.167:0/1166185227 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x56144cefb6c0 con 0x56144d2ecc00 2024-09-13T23:44:35.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a8281af40 5 note: GC not initialized 2024-09-13T23:44:35.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a817d4700 1 --2- 172.21.15.167:0/1166185227 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144d37a000 0x56144d380000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a817d4700 1 --2- 172.21.15.167:0/1166185227 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144d37a000 0x56144d380000 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x56144d27a330 tx=0x56144d202a80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a63798700 20 reqs_thread_entry: start 2024-09-13T23:44:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a8281af40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a8281af40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a61794700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:45:05 2024 2024-09-13T23:44:35.480 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a60f72700 20 reqs_thread_entry: start 2024-09-13T23:44:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a8281af40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-13T23:44:35.480 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.473+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2ed800 con 0x56144d2e0000 2024-09-13T23:44:35.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-09-13T23:44:35.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:44:35.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-09-13T23:44:35.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2edc00 con 0x56144d2e0400 2024-09-13T23:44:35.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-09-13T23:44:35.482 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-13T23:44:35.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-13T23:44:35.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-09-13T23:44:35.483 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~346 in=346b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d37a400 con 0x56144d2e0000 2024-09-13T23:44:35.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~346] v35'5 uv5 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-09-13T23:44:35.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:44:35.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-09-13T23:44:35.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.477+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94645194254592 in=534b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d37a800 con 0x56144d2e0400 2024-09-13T23:44:35.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== watch-notify(notify (1) cookie 94645192700288 notify 150323855378 ret 0) v3 ==== 564+0+0 (crc 0 0 0) 0x56144d346d00 con 0x56144d2e0400 2024-09-13T23:44:35.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a7b7c8700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855378 cookie 94645192700288 notifier 4806 bl.length()=522 2024-09-13T23:44:35.485 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(65 notify.5 [notify cookie 94645194254592 out=8b] v0'0 uv216 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a7b7c8700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-09-13T23:44:35.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a7b7c8700 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:44:35.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a7b7c8700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d3e4000 con 0x56144d2e0400 2024-09-13T23:44:35.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== watch-notify(notify_complete (2) cookie 94645194254592 notify 150323855378 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56144d346ea0 con 0x56144d2e0400 2024-09-13T23:44:35.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d37ac00 con 0x56144d2e0400 2024-09-13T23:44:35.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v35'9 uv9 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a8281af40 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-13T23:44:35.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a8281af40 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-13T23:44:35.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a8281af40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-13T23:44:35.488 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.481+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94645194254592 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d37b000 con 0x56144d242c00 2024-09-13T23:44:35.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.485+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94645192696832 notify 150323855401 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x56144cea76c0 con 0x56144d242c00 2024-09-13T23:44:35.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.485+0000 7f3a7b7c8700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855401 cookie 94645192696832 notifier 4806 bl.length()=177 2024-09-13T23:44:35.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.485+0000 7f3a7b7c8700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-09-13T23:44:35.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.485+0000 7f3a7b7c8700 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-09-13T23:44:35.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.485+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(68 notify.2 [notify cookie 94645194254592 out=8b] v0'0 uv216 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.485+0000 7f3a7b7c8700 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d3e4400 con 0x56144d242c00 2024-09-13T23:44:35.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.485+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== watch-notify(notify_complete (2) cookie 94645194254592 notify 150323855401 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x56144cea7d40 con 0x56144d242c00 2024-09-13T23:44:35.490 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.485+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.490 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:44:35.490 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "fud", 2024-09-13T23:44:35.490 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Fud", 2024-09-13T23:44:35.490 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "", 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 4, 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "fud", 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "p5YnriCv1nAtykxBrupQ", 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh" 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [], 2024-09-13T23:44:35.491 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:44:35.492 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:44:35.493 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:44:35.493 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:44:35.493 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:44:35.493 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:44:35.493 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:44:35.493 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:44:35.493 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:44:35.493 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:44:35.493 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:44:35.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.485+0000 7f3a67fa1700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:35.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.485+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94645194252288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d37b400 con 0x56144d242400 2024-09-13T23:44:35.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.489+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94645194252288] v35'215 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.489+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94645192694528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d37b800 con 0x56144d242c00 2024-09-13T23:44:35.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.493+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94645192694528] v35'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.493+0000 7f3a8281af40 20 remove_watcher() i=0 2024-09-13T23:44:35.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.493+0000 7f3a8281af40 2 removed watcher, disabling cache 2024-09-13T23:44:35.497 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.493+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94645192695680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d37bc00 con 0x56144d242400 2024-09-13T23:44:35.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.493+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94645192695680] v35'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d242400 2024-09-13T23:44:35.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.493+0000 7f3a8281af40 20 remove_watcher() i=1 2024-09-13T23:44:35.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.493+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94645192696832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2ec000 con 0x56144d242c00 2024-09-13T23:44:35.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.497+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94645192696832] v35'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.497+0000 7f3a8281af40 20 remove_watcher() i=2 2024-09-13T23:44:35.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.497+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94645192697984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2ec400 con 0x56144d2e0000 2024-09-13T23:44:35.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.501+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94645192697984] v35'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.501+0000 7f3a8281af40 20 remove_watcher() i=3 2024-09-13T23:44:35.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.501+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94645192699136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2ed800 con 0x56144d242c00 2024-09-13T23:44:35.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.501+0000 7f3a80fd3700 1 -- 172.21.15.167:0/2790292812 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94645192699136] v35'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd200 con 0x56144d242c00 2024-09-13T23:44:35.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.501+0000 7f3a8281af40 20 remove_watcher() i=4 2024-09-13T23:44:35.508 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.501+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94645192700288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e1c00 con 0x56144d2e0400 2024-09-13T23:44:35.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.505+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94645192700288] v35'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.505+0000 7f3a8281af40 20 remove_watcher() i=5 2024-09-13T23:44:35.510 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.505+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94645192701440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d2e0c00 con 0x56144d2e0400 2024-09-13T23:44:35.512 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94645192701440] v35'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dd440 con 0x56144d2e0400 2024-09-13T23:44:35.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 20 remove_watcher() i=6 2024-09-13T23:44:35.513 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94645192702592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x56144d3e4000 con 0x56144d2e0000 2024-09-13T23:44:35.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a817d4700 1 -- 172.21.15.167:0/2790292812 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94645192702592] v35'218 uv216 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x56144d2dcb40 con 0x56144d2e0000 2024-09-13T23:44:35.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 20 remove_watcher() i=7 2024-09-13T23:44:35.516 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56144d242c00 msgr2=0x56144d17c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x56144d242c00 0x56144d17c100 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56144d242400 msgr2=0x56144d17bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x56144d242400 0x56144d17bb80 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56144d2e0000 msgr2=0x56144d17a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x56144d2e0000 0x56144d17a580 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56144d2e0400 msgr2=0x56144d17cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x56144d2e0400 0x56144d17cc00 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144d243800 msgr2=0x56144d17c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144d243800 0x56144d17c680 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x56144dc68030 tx=0x56144d2cc000 comp rx=0 tx=0).stop 2024-09-13T23:44:35.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 msgr2=0x56144d17b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a807d2700 1 -- 172.21.15.167:0/2790292812 reap_dead start 2024-09-13T23:44:35.517 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 0x56144d17b600 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x56144d1c21b0 tx=0x56144d202e40 comp rx=0 tx=0).stop 2024-09-13T23:44:35.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 shutdown_connections 2024-09-13T23:44:35.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2790292812 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d242000 0x56144d17b600 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 >> 172.21.15.167:0/2790292812 conn(0x56144ce95800 msgr2=0x56144cf95800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 shutdown_connections 2024-09-13T23:44:35.518 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.509+0000 7f3a8281af40 1 -- 172.21.15.167:0/2790292812 wait complete. 2024-09-13T23:44:35.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 -- 172.21.15.167:0/1166185227 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144d37a000 msgr2=0x56144d380000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 --2- 172.21.15.167:0/1166185227 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144d37a000 0x56144d380000 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x56144d27a330 tx=0x56144d202a80 comp rx=0 tx=0).stop 2024-09-13T23:44:35.519 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 -- 172.21.15.167:0/1166185227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 msgr2=0x56144d17d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 --2- 172.21.15.167:0/1166185227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 0x56144d17d180 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x56144d2f6240 tx=0x56144d2cc0c0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 -- 172.21.15.167:0/1166185227 shutdown_connections 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 --2- 172.21.15.167:0/1166185227 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144d37a000 0x56144d380000 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 --2- 172.21.15.167:0/1166185227 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144d2ecc00 0x56144d17d180 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 -- 172.21.15.167:0/1166185227 >> 172.21.15.167:0/1166185227 conn(0x56144d2ec800 msgr2=0x56144cf96000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 -- 172.21.15.167:0/1166185227 shutdown_connections 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 -- 172.21.15.167:0/1166185227 wait complete. 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 -- 172.21.15.167:0/2236064158 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144ce95c00 msgr2=0x56144d17b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2236064158 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144ce95c00 0x56144d17b080 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x56144d1c2300 tx=0x56144d202b40 comp rx=0 tx=0).stop 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 -- 172.21.15.167:0/2236064158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95400 msgr2=0x56144d17ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2236064158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95400 0x56144d17ab00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x56144dc69f50 tx=0x56144cf99920 comp rx=0 tx=0).stop 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 -- 172.21.15.167:0/2236064158 shutdown_connections 2024-09-13T23:44:35.520 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2236064158 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x56144ce95c00 0x56144d17b080 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 --2- 172.21.15.167:0/2236064158 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x56144ce95400 0x56144d17ab00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 -- 172.21.15.167:0/2236064158 >> 172.21.15.167:0/2236064158 conn(0x56144ce94c00 msgr2=0x56144cf95000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 -- 172.21.15.167:0/2236064158 shutdown_connections 2024-09-13T23:44:35.521 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.513+0000 7f3a8281af40 1 -- 172.21.15.167:0/2236064158 wait complete. 2024-09-13T23:44:35.522 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:44:35.582 INFO:tasks.radosgw_admin:RL: create_bucket myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:44:35.592 INFO:tasks.radosgw_admin:RL: put_obj myfoo7 fud bytes_out=8 bytes_in=0 failed=False 2024-09-13T23:44:35.592 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-09-13T23:44:35.592 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-09-13T23:44:35.592 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bud --new-uid fud 2024-09-13T23:44:35.632 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:35.632 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:35.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eedb2ff40 1 Processor -- start 2024-09-13T23:44:35.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eedb2ff40 1 -- start start 2024-09-13T23:44:35.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eedb2ff40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ba8a1a1400 0x55ba8a486580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eedb2ff40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55ba8a265500 con 0x55ba8a1a1000 2024-09-13T23:44:35.633 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eedb2ff40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55ba8a22ea00 con 0x55ba8a1a1400 2024-09-13T23:44:35.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eebae7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ba8a1a1400 0x55ba8a486580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eec2e8700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55ba8a1a1000 0x55ba8a2a0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:36462/0 (socket says 172.21.15.167:36462) 2024-09-13T23:44:35.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eebae7700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55ba8a1a1400 0x55ba8a486580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42980/0 (socket says 172.21.15.167:42980) 2024-09-13T23:44:35.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eec2e8700 1 -- 172.21.15.167:0/1587177415 learned_addr learned my addr 172.21.15.167:0/1587177415 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:44:35.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eebae7700 1 -- 172.21.15.167:0/1587177415 >> v1:172.21.15.12:6789/0 conn(0x55ba8a1a1000 legacy=0x55ba8a2a0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eebae7700 1 -- 172.21.15.167:0/1587177415 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8a1b3d40 con 0x55ba8a1a1400 2024-09-13T23:44:35.634 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eeb2e6700 1 -- 172.21.15.167:0/1587177415 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970483369 0 0) 0x55ba8a2fc000 con 0x55ba8a1a1000 2024-09-13T23:44:35.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eebae7700 1 --2- 172.21.15.167:0/1587177415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1400 0x55ba8a486580 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x55ba8a481920 tx=0x55ba8a2a5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb06815d0432081 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eeb2e6700 1 -- 172.21.15.167:0/1587177415 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a2454a0 con 0x55ba8a1a1400 2024-09-13T23:44:35.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eeb2e6700 1 -- 172.21.15.167:0/1587177415 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba8a265800 con 0x55ba8a1a1400 2024-09-13T23:44:35.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eeb2e6700 1 -- 172.21.15.167:0/1587177415 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a245600 con 0x55ba8a1a1400 2024-09-13T23:44:35.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1587177415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1400 msgr2=0x55ba8a486580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/1587177415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1400 0x55ba8a486580 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x55ba8a481920 tx=0x55ba8a2a5920 comp rx=0 tx=0).stop 2024-09-13T23:44:35.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1587177415 shutdown_connections 2024-09-13T23:44:35.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/1587177415 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1400 0x55ba8a486580 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1587177415 >> 172.21.15.167:0/1587177415 conn(0x55ba8a1a0c00 msgr2=0x55ba8a2a0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1587177415 shutdown_connections 2024-09-13T23:44:35.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.629+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1587177415 wait complete. 2024-09-13T23:44:35.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 Processor -- start 2024-09-13T23:44:35.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 -- start start 2024-09-13T23:44:35.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 0x55ba8a486580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8a22e500 con 0x55ba8a1a1000 2024-09-13T23:44:35.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eec2e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 0x55ba8a486580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eec2e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 0x55ba8a486580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42992/0 (socket says 172.21.15.167:42992) 2024-09-13T23:44:35.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eec2e8700 1 -- 172.21.15.167:0/1303530218 learned_addr learned my addr 172.21.15.167:0/1303530218 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eec2e8700 1 -- 172.21.15.167:0/1303530218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8a1b3ba0 con 0x55ba8a1a1000 2024-09-13T23:44:35.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eec2e8700 1 --2- 172.21.15.167:0/1303530218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 0x55ba8a486580 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x55ba8af75d70 tx=0x55ba8a2a5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed243c481768c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eeaae5700 1 -- 172.21.15.167:0/1303530218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a2451e0 con 0x55ba8a1a1000 2024-09-13T23:44:35.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eeaae5700 1 -- 172.21.15.167:0/1303530218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba8a265b00 con 0x55ba8a1a1000 2024-09-13T23:44:35.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eeaae5700 1 -- 172.21.15.167:0/1303530218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a2454a0 con 0x55ba8a1a1000 2024-09-13T23:44:35.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1303530218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 msgr2=0x55ba8a486580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/1303530218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 0x55ba8a486580 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x55ba8af75d70 tx=0x55ba8a2a5b60 comp rx=0 tx=0).stop 2024-09-13T23:44:35.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1303530218 shutdown_connections 2024-09-13T23:44:35.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/1303530218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 0x55ba8a486580 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1303530218 >> 172.21.15.167:0/1303530218 conn(0x55ba8a1a0c00 msgr2=0x55ba8a2a1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1303530218 shutdown_connections 2024-09-13T23:44:35.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1303530218 wait complete. 2024-09-13T23:44:35.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 Processor -- start 2024-09-13T23:44:35.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 -- start start 2024-09-13T23:44:35.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 0x55ba8a486580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.633+0000 7f5eedb2ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8a22e3c0 con 0x55ba8a1a1000 2024-09-13T23:44:35.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5eec2e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 0x55ba8a486580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5eec2e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 0x55ba8a486580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42996/0 (socket says 172.21.15.167:42996) 2024-09-13T23:44:35.641 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5eec2e8700 1 -- 172.21.15.167:0/2353719248 learned_addr learned my addr 172.21.15.167:0/2353719248 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5eec2e8700 1 -- 172.21.15.167:0/2353719248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8a1b3a00 con 0x55ba8a1a1000 2024-09-13T23:44:35.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5eec2e8700 1 --2- 172.21.15.167:0/2353719248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 0x55ba8a486580 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x55ba8af75e00 tx=0x55ba8a2a5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5ee92e2700 1 -- 172.21.15.167:0/2353719248 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a245600 con 0x55ba8a1a1000 2024-09-13T23:44:35.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5ee92e2700 1 -- 172.21.15.167:0/2353719248 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba8a265c80 con 0x55ba8a1a1000 2024-09-13T23:44:35.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5ee92e2700 1 -- 172.21.15.167:0/2353719248 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a2451e0 con 0x55ba8a1a1000 2024-09-13T23:44:35.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/2353719248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ba8a1b3860 con 0x55ba8a1a1000 2024-09-13T23:44:35.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/2353719248 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba8a1b36c0 con 0x55ba8a1a1000 2024-09-13T23:44:35.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5ee92e2700 1 -- 172.21.15.167:0/2353719248 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ba8a29ab00 con 0x55ba8a1a1000 2024-09-13T23:44:35.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5eedb2ff40 1 Processor -- start 2024-09-13T23:44:35.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5ee92e2700 1 --2- 172.21.15.167:0/2353719248 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a54c000 0x55ba8a487080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5ee92e2700 1 -- 172.21.15.167:0/2353719248 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55ba8a207180 con 0x55ba8a1a1000 2024-09-13T23:44:35.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5eedb2ff40 1 -- start start 2024-09-13T23:44:35.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5eecae9700 1 --2- 172.21.15.167:0/2353719248 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a54c000 0x55ba8a487080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5eedb2ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 0x55ba8a486b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.637+0000 7f5eedb2ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8a1c6640 con 0x55ba8a1a1800 2024-09-13T23:44:35.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eecae9700 1 --2- 172.21.15.167:0/2353719248 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a54c000 0x55ba8a487080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55ba8a4d0e10 tx=0x55ba8a504de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eec2e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 0x55ba8a486b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eec2e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 0x55ba8a486b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43010/0 (socket says 172.21.15.167:43010) 2024-09-13T23:44:35.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eec2e8700 1 -- 172.21.15.167:0/494953485 learned_addr learned my addr 172.21.15.167:0/494953485 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eec2e8700 1 -- 172.21.15.167:0/494953485 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8a1b2000 con 0x55ba8a1a1800 2024-09-13T23:44:35.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eec2e8700 1 --2- 172.21.15.167:0/494953485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 0x55ba8a486b00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55ba8a4d1da0 tx=0x55ba8a504240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304e1a2b6df887ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5ee6add700 1 -- 172.21.15.167:0/494953485 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a2454a0 con 0x55ba8a1a1800 2024-09-13T23:44:35.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5ee6add700 1 -- 172.21.15.167:0/494953485 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba8a265c80 con 0x55ba8a1a1800 2024-09-13T23:44:35.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5ee6add700 1 -- 172.21.15.167:0/494953485 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a2458c0 con 0x55ba8a1a1800 2024-09-13T23:44:35.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/494953485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 msgr2=0x55ba8a486b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/494953485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 0x55ba8a486b00 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x55ba8a4d1da0 tx=0x55ba8a504240 comp rx=0 tx=0).stop 2024-09-13T23:44:35.648 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/494953485 shutdown_connections 2024-09-13T23:44:35.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/494953485 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 0x55ba8a486b00 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/494953485 >> 172.21.15.167:0/494953485 conn(0x55ba8a1a1400 msgr2=0x55ba8a2a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/494953485 shutdown_connections 2024-09-13T23:44:35.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.641+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/494953485 wait complete. 2024-09-13T23:44:35.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5eedb2ff40 1 Processor -- start 2024-09-13T23:44:35.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5eedb2ff40 1 -- start start 2024-09-13T23:44:35.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5eedb2ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 0x55ba8a486b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5eedb2ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8a1c68c0 con 0x55ba8a1a1800 2024-09-13T23:44:35.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5eec2e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 0x55ba8a486b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5eec2e8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 0x55ba8a486b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43012/0 (socket says 172.21.15.167:43012) 2024-09-13T23:44:35.650 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 learned_addr learned my addr 172.21.15.167:0/4185287416 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8a1b2ea0 con 0x55ba8a1a1800 2024-09-13T23:44:35.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5eec2e8700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 0x55ba8a486b00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x55ba8a568c30 tx=0x55ba8a504fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5ee52da700 1 -- 172.21.15.167:0/4185287416 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a2454a0 con 0x55ba8a1a1800 2024-09-13T23:44:35.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5ee52da700 1 -- 172.21.15.167:0/4185287416 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba8a56e000 con 0x55ba8a1a1800 2024-09-13T23:44:35.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5ee52da700 1 -- 172.21.15.167:0/4185287416 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a2458c0 con 0x55ba8a1a1800 2024-09-13T23:44:35.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ba8a1b2820 con 0x55ba8a1a1800 2024-09-13T23:44:35.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.645+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba8a1b29c0 con 0x55ba8a1a1800 2024-09-13T23:44:35.653 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5ee52da700 1 -- 172.21.15.167:0/4185287416 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ba8a29adc0 con 0x55ba8a1a1800 2024-09-13T23:44:35.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5ee52da700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a54d000 0x55ba8a487600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5ee52da700 1 -- 172.21.15.167:0/4185287416 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55ba8a207340 con 0x55ba8a1a1800 2024-09-13T23:44:35.654 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5eebae7700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a54d000 0x55ba8a487600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5eedb2ff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ba8a54d400 0x55ba8a487b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a54d800 con 0x55ba8a54d400 2024-09-13T23:44:35.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5eecae9700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ba8a54d400 0x55ba8a487b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5eebae7700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a54d000 0x55ba8a487600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55ba8a481920 tx=0x55ba8a2a5800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5eecae9700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ba8a54d400 0x55ba8a487b80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5eedb2ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5eedb2ff40 20 realm 2024-09-13T23:44:35.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5eedb2ff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.656 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.649+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a54dc00 con 0x55ba8a54d400 2024-09-13T23:44:35.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:35.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a54d800 con 0x55ba8a54d400 2024-09-13T23:44:35.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ba8a54dc00 0x55ba8a488100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8000 con 0x55ba8a54dc00 2024-09-13T23:44:35.658 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eec2e8700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ba8a54dc00 0x55ba8a488100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eec2e8700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ba8a54dc00 0x55ba8a488100 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:35.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54dc00 2024-09-13T23:44:35.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:35.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 20 searching for the correct realm 2024-09-13T23:44:35.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ba8a5e8800 0x55ba8a488680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.660 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8c00 con 0x55ba8a5e8800 2024-09-13T23:44:35.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.653+0000 7f5eebae7700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ba8a5e8800 0x55ba8a488680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eebae7700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ba8a5e8800 0x55ba8a488680 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.661 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8c00 con 0x55ba8a54dc00 2024-09-13T23:44:35.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eec2e8700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ba8a5e8000 0x55ba8a488c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8000 2024-09-13T23:44:35.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eecae9700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ba8a5e8000 0x55ba8a488c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eecae9700 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ba8a5e8000 0x55ba8a488c00 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54d400 2024-09-13T23:44:35.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54d400 2024-09-13T23:44:35.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.657+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8800 2024-09-13T23:44:35.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.661+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.661+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54d400 2024-09-13T23:44:35.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.661+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.661+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54dc00 2024-09-13T23:44:35.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.661+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.661+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54dc00 2024-09-13T23:44:35.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.661+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.661+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54d400 2024-09-13T23:44:35.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.661+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.661+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8000 2024-09-13T23:44:35.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.661+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.661+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54d400 2024-09-13T23:44:35.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.665+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.665+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8000 2024-09-13T23:44:35.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.665+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.665+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54dc00 2024-09-13T23:44:35.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.665+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.665+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8800 2024-09-13T23:44:35.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.665+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.665+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54d400 2024-09-13T23:44:35.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.665+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.665+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54dc00 2024-09-13T23:44:35.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.665+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8800 2024-09-13T23:44:35.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54d400 2024-09-13T23:44:35.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8000 2024-09-13T23:44:35.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54d400 2024-09-13T23:44:35.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8800 2024-09-13T23:44:35.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8000 2024-09-13T23:44:35.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.669+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54d400 2024-09-13T23:44:35.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.673+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.673+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8000 2024-09-13T23:44:35.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.673+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.673+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8800 2024-09-13T23:44:35.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.673+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.673+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8000 2024-09-13T23:44:35.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.673+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.673+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54dc00 2024-09-13T23:44:35.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.673+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.673+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54dc00 2024-09-13T23:44:35.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.673+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.673+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8800 2024-09-13T23:44:35.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54dc00 2024-09-13T23:44:35.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54dc00 2024-09-13T23:44:35.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eedb2ff40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:35.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eedb2ff40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:35.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eedb2ff40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:35.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eedb2ff40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:35.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eedb2ff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9000 con 0x55ba8a54d400 2024-09-13T23:44:35.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eedb2ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eedb2ff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9400 con 0x55ba8a54dc00 2024-09-13T23:44:35.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eedb2ff40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:35.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eedb2ff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.677+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9800 con 0x55ba8a54dc00 2024-09-13T23:44:35.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:35.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 20 zone default found 2024-09-13T23:44:35.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 4 Realm: () 2024-09-13T23:44:35.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:35.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:35.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:35.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 20 zonegroup default 2024-09-13T23:44:35.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9c00 con 0x55ba8a5e8000 2024-09-13T23:44:35.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9000 con 0x55ba8a5e8800 2024-09-13T23:44:35.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9c00 con 0x55ba8a5e8800 2024-09-13T23:44:35.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 20 started sync module instance, tier type = 2024-09-13T23:44:35.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:35.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.681+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a54d800 con 0x55ba8a54dc00 2024-09-13T23:44:35.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.685+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.685+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94259672242944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8c00 con 0x55ba8a54dc00 2024-09-13T23:44:35.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.685+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a54d400 2024-09-13T23:44:35.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.689+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.689+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94259672244096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9400 con 0x55ba8a54d400 2024-09-13T23:44:35.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.689+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94259672242944] v35'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.694 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.689+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9800 con 0x55ba8a54dc00 2024-09-13T23:44:35.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.689+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94259672244096] v35'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.693+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.693+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94259672245248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9000 con 0x55ba8a54dc00 2024-09-13T23:44:35.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.693+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9c00 con 0x55ba8a5e8000 2024-09-13T23:44:35.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.693+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94259672245248] v35'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.693+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.693+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94259672246400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a54c800 con 0x55ba8a5e8000 2024-09-13T23:44:35.700 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.693+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a54d800 con 0x55ba8a54dc00 2024-09-13T23:44:35.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.697+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94259672246400] v35'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.702 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.697+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.697+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94259672247552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a54c800 con 0x55ba8a54dc00 2024-09-13T23:44:35.703 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.697+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8800 2024-09-13T23:44:35.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.701+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94259672247552] v35'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.701+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.701+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94259672248704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9400 con 0x55ba8a5e8800 2024-09-13T23:44:35.705 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.701+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9c00 con 0x55ba8a5e8800 2024-09-13T23:44:35.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.701+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(56 notify.6 [create] v35'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.701+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94259672249856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e8400 con 0x55ba8a5e8800 2024-09-13T23:44:35.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.701+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9400 con 0x55ba8a5e8000 2024-09-13T23:44:35.707 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.701+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(55 notify.5 [watch watch cookie 94259672248704] v35'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94259672249856] v35'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'219 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94259672251008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5e9c00 con 0x55ba8a5e8000 2024-09-13T23:44:35.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eedb2ff40 20 add_watcher() i=0 2024-09-13T23:44:35.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eedb2ff40 20 add_watcher() i=1 2024-09-13T23:44:35.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eedb2ff40 20 add_watcher() i=2 2024-09-13T23:44:35.709 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eedb2ff40 20 add_watcher() i=3 2024-09-13T23:44:35.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eedb2ff40 20 add_watcher() i=4 2024-09-13T23:44:35.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eedb2ff40 20 add_watcher() i=5 2024-09-13T23:44:35.710 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eedb2ff40 20 add_watcher() i=6 2024-09-13T23:44:35.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94259672251008] v35'220 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eedb2ff40 20 add_watcher() i=7 2024-09-13T23:44:35.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eedb2ff40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:35.711 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5f6000 con 0x55ba8a54d400 2024-09-13T23:44:35.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.712 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.705+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94259673800704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5f6400 con 0x55ba8a54d400 2024-09-13T23:44:35.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.709+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94259673800704] v35'216 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.709+0000 7f5ed32b6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:35.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.709+0000 7f5eedb2ff40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:35.714 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.709+0000 7f5eedb2ff40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:35.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.709+0000 7f5eedb2ff40 1 Processor -- start 2024-09-13T23:44:35.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.709+0000 7f5eedb2ff40 1 -- start start 2024-09-13T23:44:35.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.709+0000 7f5eedb2ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 0x55ba8a489180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.715 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.709+0000 7f5eedb2ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8a22e3c0 con 0x55ba8a5f6c00 2024-09-13T23:44:35.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.709+0000 7f5eebae7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 0x55ba8a489180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.709+0000 7f5eebae7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 0x55ba8a489180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43022/0 (socket says 172.21.15.167:43022) 2024-09-13T23:44:35.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.709+0000 7f5eebae7700 1 -- 172.21.15.167:0/1436377902 learned_addr learned my addr 172.21.15.167:0/1436377902 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.709+0000 7f5eebae7700 1 -- 172.21.15.167:0/1436377902 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8a1b2b60 con 0x55ba8a5f6c00 2024-09-13T23:44:35.716 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eebae7700 1 --2- 172.21.15.167:0/1436377902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 0x55ba8a489180 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x55ba8a587d70 tx=0x55ba8a2a5860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d840480f6778314a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5ed2ab5700 1 -- 172.21.15.167:0/1436377902 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a245760 con 0x55ba8a5f6c00 2024-09-13T23:44:35.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5ed2ab5700 1 -- 172.21.15.167:0/1436377902 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba8a56e900 con 0x55ba8a5f6c00 2024-09-13T23:44:35.717 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5ed2ab5700 1 -- 172.21.15.167:0/1436377902 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a5f8160 con 0x55ba8a5f6c00 2024-09-13T23:44:35.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1436377902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 msgr2=0x55ba8a489180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/1436377902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 0x55ba8a489180 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x55ba8a587d70 tx=0x55ba8a2a5860 comp rx=0 tx=0).stop 2024-09-13T23:44:35.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1436377902 shutdown_connections 2024-09-13T23:44:35.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/1436377902 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 0x55ba8a489180 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1436377902 >> 172.21.15.167:0/1436377902 conn(0x55ba8a5f6800 msgr2=0x55ba8a2a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1436377902 shutdown_connections 2024-09-13T23:44:35.718 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1436377902 wait complete. 2024-09-13T23:44:35.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eedb2ff40 1 Processor -- start 2024-09-13T23:44:35.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eedb2ff40 1 -- start start 2024-09-13T23:44:35.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eedb2ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 0x55ba8a489180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eedb2ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba8a22e500 con 0x55ba8a5f6c00 2024-09-13T23:44:35.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eebae7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 0x55ba8a489180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eebae7700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 0x55ba8a489180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43028/0 (socket says 172.21.15.167:43028) 2024-09-13T23:44:35.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eebae7700 1 -- 172.21.15.167:0/1982825811 learned_addr learned my addr 172.21.15.167:0/1982825811 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.720 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.713+0000 7f5eebae7700 1 -- 172.21.15.167:0/1982825811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba8a1b2d00 con 0x55ba8a5f6c00 2024-09-13T23:44:35.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.717+0000 7f5eebae7700 1 --2- 172.21.15.167:0/1982825811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 0x55ba8a489180 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x55ba8a6021b0 tx=0x55ba8a5c6420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.717+0000 7f5ed12b2700 1 -- 172.21.15.167:0/1982825811 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a5f82c0 con 0x55ba8a5f6c00 2024-09-13T23:44:35.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.717+0000 7f5ed12b2700 1 -- 172.21.15.167:0/1982825811 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ba8a56e600 con 0x55ba8a5f6c00 2024-09-13T23:44:35.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.717+0000 7f5ed12b2700 1 -- 172.21.15.167:0/1982825811 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55ba8a5f8420 con 0x55ba8a5f6c00 2024-09-13T23:44:35.721 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.717+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1982825811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ba8a1b2340 con 0x55ba8a5f6c00 2024-09-13T23:44:35.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.717+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1982825811 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba8a1b3d40 con 0x55ba8a5f6c00 2024-09-13T23:44:35.722 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.717+0000 7f5ed12b2700 1 -- 172.21.15.167:0/1982825811 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55ba8a29bb80 con 0x55ba8a5f6c00 2024-09-13T23:44:35.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.717+0000 7f5ed12b2700 1 --2- 172.21.15.167:0/1982825811 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a682000 0x55ba8a688000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.717+0000 7f5ed12b2700 1 -- 172.21.15.167:0/1982825811 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55ba8a2076c0 con 0x55ba8a5f6c00 2024-09-13T23:44:35.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.717+0000 7f5eecae9700 1 --2- 172.21.15.167:0/1982825811 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a682000 0x55ba8a688000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.717+0000 7f5eedb2ff40 5 note: GC not initialized 2024-09-13T23:44:35.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.717+0000 7f5eecae9700 1 --2- 172.21.15.167:0/1982825811 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a682000 0x55ba8a688000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55ba8a569440 tx=0x55ba8a504d20 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eceaad700 20 reqs_thread_entry: start 2024-09-13T23:44:35.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eedb2ff40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:35.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eedb2ff40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:35.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eccaa9700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:45:06 2024 2024-09-13T23:44:35.726 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:35.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5ecc287700 20 reqs_thread_entry: start 2024-09-13T23:44:35.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eedb2ff40 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-09-13T23:44:35.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eedb2ff40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5f7800 con 0x55ba8a5e8000 2024-09-13T23:44:35.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(62 bud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eedb2ff40 20 rados_obj.operate() r=0 bl.length=346 2024-09-13T23:44:35.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eedb2ff40 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-09-13T23:44:35.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eedb2ff40 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-09-13T23:44:35.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eedb2ff40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:44:35.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.721+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.e 8:75f48285:users.uid::fud:head [create,call version.set in=58b,writefull 0~181 in=181b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5f7c00 con 0x55ba8a5e8000 2024-09-13T23:44:35.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.725+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(63 fud [create,call,writefull 0~181] v35'6 uv5 ondisk = -17 ((17) File exists)) v8 ==== 231+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.729 INFO:teuthology.orchestra.run.smithi167.stderr:could not rename user: unable to rename user, user name given by --new-uid already exists. to overwrite this user, add --yes-i-really-mean-it 2024-09-13T23:44:35.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.725+0000 7f5ed32b6700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:35.731 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.725+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94259673800704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a682400 con 0x55ba8a54d400 2024-09-13T23:44:35.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.729+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94259673800704] v35'217 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.733 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.729+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94259672242944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a682800 con 0x55ba8a54dc00 2024-09-13T23:44:35.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.733+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94259672242944] v35'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.733+0000 7f5eedb2ff40 20 remove_watcher() i=0 2024-09-13T23:44:35.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.733+0000 7f5eedb2ff40 2 removed watcher, disabling cache 2024-09-13T23:44:35.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.733+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94259672244096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a682c00 con 0x55ba8a54d400 2024-09-13T23:44:35.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.733+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94259672244096] v35'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a54d400 2024-09-13T23:44:35.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.733+0000 7f5eedb2ff40 20 remove_watcher() i=1 2024-09-13T23:44:35.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.733+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94259672245248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a683000 con 0x55ba8a54dc00 2024-09-13T23:44:35.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.737+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94259672245248] v35'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.741 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.737+0000 7f5eedb2ff40 20 remove_watcher() i=2 2024-09-13T23:44:35.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.737+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94259672246400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a683400 con 0x55ba8a5e8000 2024-09-13T23:44:35.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.737+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94259672246400] v35'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.737+0000 7f5eedb2ff40 20 remove_watcher() i=3 2024-09-13T23:44:35.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.737+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94259672247552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a683800 con 0x55ba8a54dc00 2024-09-13T23:44:35.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.741+0000 7f5eec2e8700 1 -- 172.21.15.167:0/4185287416 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94259672247552] v35'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3200 con 0x55ba8a54dc00 2024-09-13T23:44:35.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.741+0000 7f5eedb2ff40 20 remove_watcher() i=4 2024-09-13T23:44:35.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.741+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94259672248704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a683c00 con 0x55ba8a5e8800 2024-09-13T23:44:35.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.745+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94259672248704] v35'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.745+0000 7f5eedb2ff40 20 remove_watcher() i=5 2024-09-13T23:44:35.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.745+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94259672249856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5f6000 con 0x55ba8a5e8800 2024-09-13T23:44:35.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.745+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94259672249856] v35'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e3440 con 0x55ba8a5e8800 2024-09-13T23:44:35.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.745+0000 7f5eedb2ff40 20 remove_watcher() i=6 2024-09-13T23:44:35.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.745+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94259672251008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55ba8a5f6400 con 0x55ba8a5e8000 2024-09-13T23:44:35.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eecae9700 1 -- 172.21.15.167:0/4185287416 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94259672251008] v35'221 uv219 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55ba8a5e2b40 con 0x55ba8a5e8000 2024-09-13T23:44:35.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 20 remove_watcher() i=7 2024-09-13T23:44:35.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ba8a54dc00 msgr2=0x55ba8a488100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55ba8a54dc00 0x55ba8a488100 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ba8a54d400 msgr2=0x55ba8a487b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55ba8a54d400 0x55ba8a487b80 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ba8a5e8000 msgr2=0x55ba8a488c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55ba8a5e8000 0x55ba8a488c00 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ba8a5e8800 msgr2=0x55ba8a488680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55ba8a5e8800 0x55ba8a488680 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a54d000 msgr2=0x55ba8a487600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a54d000 0x55ba8a487600 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55ba8a481920 tx=0x55ba8a2a5800 comp rx=0 tx=0).stop 2024-09-13T23:44:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 msgr2=0x55ba8a486b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/4185287416 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1800 0x55ba8a486b00 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x55ba8a568c30 tx=0x55ba8a504fc0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 reap_dead start 2024-09-13T23:44:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eebae7700 1 -- 172.21.15.167:0/4185287416 reap_dead start 2024-09-13T23:44:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 shutdown_connections 2024-09-13T23:44:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 >> 172.21.15.167:0/4185287416 conn(0x55ba8a1a1400 msgr2=0x55ba8a2a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 shutdown_connections 2024-09-13T23:44:35.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/4185287416 wait complete. 2024-09-13T23:44:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1982825811 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a682000 msgr2=0x55ba8a688000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/1982825811 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a682000 0x55ba8a688000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55ba8a569440 tx=0x55ba8a504d20 comp rx=0 tx=0).stop 2024-09-13T23:44:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1982825811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 msgr2=0x55ba8a489180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.749+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/1982825811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 0x55ba8a489180 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x55ba8a6021b0 tx=0x55ba8a5c6420 comp rx=0 tx=0).stop 2024-09-13T23:44:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1982825811 shutdown_connections 2024-09-13T23:44:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/1982825811 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a682000 0x55ba8a688000 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/1982825811 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a5f6c00 0x55ba8a489180 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1982825811 >> 172.21.15.167:0/1982825811 conn(0x55ba8a5f6800 msgr2=0x55ba8a2a2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1982825811 shutdown_connections 2024-09-13T23:44:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/1982825811 wait complete. 2024-09-13T23:44:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/2353719248 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a54c000 msgr2=0x55ba8a487080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/2353719248 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a54c000 0x55ba8a487080 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x55ba8a4d0e10 tx=0x55ba8a504de0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/2353719248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 msgr2=0x55ba8a486580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/2353719248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 0x55ba8a486580 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x55ba8af75e00 tx=0x55ba8a2a5a40 comp rx=0 tx=0).stop 2024-09-13T23:44:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/2353719248 shutdown_connections 2024-09-13T23:44:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/2353719248 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55ba8a54c000 0x55ba8a487080 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 --2- 172.21.15.167:0/2353719248 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55ba8a1a1000 0x55ba8a486580 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/2353719248 >> 172.21.15.167:0/2353719248 conn(0x55ba8a1a0c00 msgr2=0x55ba8a2a1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/2353719248 shutdown_connections 2024-09-13T23:44:35.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.753+0000 7f5eedb2ff40 1 -- 172.21.15.167:0/2353719248 wait complete. 2024-09-13T23:44:35.763 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-09-13T23:44:35.813 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:44:35.813 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-09-13T23:44:35.822 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-09-13T23:44:35.822 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=8 failed=False 2024-09-13T23:44:35.822 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud', '--purge-data'] 2024-09-13T23:44:35.822 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud', '--purge-data'] 2024-09-13T23:44:35.823 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud --purge-data 2024-09-13T23:44:35.863 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:35.863 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.857+0000 7f8eb4d73f40 1 Processor -- start 2024-09-13T23:44:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 -- start start 2024-09-13T23:44:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55966f8a5000 0x55966fb8a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55966f969500 con 0x55966f8a5400 2024-09-13T23:44:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55966f932a00 con 0x55966f8a5000 2024-09-13T23:44:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb352c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55966f8a5000 0x55966fb8a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.865 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb2d2b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55966f8a5400 0x55966f9a4800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:36470/0 (socket says 172.21.15.167:36470) 2024-09-13T23:44:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/1849151611 learned_addr learned my addr 172.21.15.167:0/1849151611 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:44:35.866 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb352c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55966f8a5000 0x55966fb8a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43034/0 (socket says 172.21.15.167:43034) 2024-09-13T23:44:35.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb352c700 1 -- 172.21.15.167:0/1849151611 >> v1:172.21.15.12:6789/0 conn(0x55966f8a5400 legacy=0x55966f9a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb352c700 1 -- 172.21.15.167:0/1849151611 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55966f8b7d40 con 0x55966f8a5000 2024-09-13T23:44:35.867 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb352c700 1 --2- 172.21.15.167:0/1849151611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x55966fb85a70 tx=0x55966f9a9920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1810e140182536d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb252a700 1 -- 172.21.15.167:0/1849151611 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966f9494a0 con 0x55966f8a5000 2024-09-13T23:44:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb252a700 1 -- 172.21.15.167:0/1849151611 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55966f969800 con 0x55966f8a5000 2024-09-13T23:44:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb252a700 1 -- 172.21.15.167:0/1849151611 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966f949600 con 0x55966f8a5000 2024-09-13T23:44:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1849151611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 msgr2=0x55966fb8a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1849151611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x55966fb85a70 tx=0x55966f9a9920 comp rx=0 tx=0).stop 2024-09-13T23:44:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1849151611 shutdown_connections 2024-09-13T23:44:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1849151611 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1849151611 >> 172.21.15.167:0/1849151611 conn(0x55966f8a4c00 msgr2=0x55966f9a4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1849151611 shutdown_connections 2024-09-13T23:44:35.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1849151611 wait complete. 2024-09-13T23:44:35.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 Processor -- start 2024-09-13T23:44:35.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.861+0000 7f8eb4d73f40 1 -- start start 2024-09-13T23:44:35.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966f932500 con 0x55966f8a5000 2024-09-13T23:44:35.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb352c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb352c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43042/0 (socket says 172.21.15.167:43042) 2024-09-13T23:44:35.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb352c700 1 -- 172.21.15.167:0/1085469684 learned_addr learned my addr 172.21.15.167:0/1085469684 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb352c700 1 -- 172.21.15.167:0/1085469684 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55966f8b7ba0 con 0x55966f8a5000 2024-09-13T23:44:35.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb352c700 1 --2- 172.21.15.167:0/1085469684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x559670709860 tx=0x55966f9a9800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c8e2948c04cca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb1d29700 1 -- 172.21.15.167:0/1085469684 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966f949760 con 0x55966f8a5000 2024-09-13T23:44:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb1d29700 1 -- 172.21.15.167:0/1085469684 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55966f969b00 con 0x55966f8a5000 2024-09-13T23:44:35.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb1d29700 1 -- 172.21.15.167:0/1085469684 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966f9494a0 con 0x55966f8a5000 2024-09-13T23:44:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1085469684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 msgr2=0x55966fb8a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1085469684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x559670709860 tx=0x55966f9a9800 comp rx=0 tx=0).stop 2024-09-13T23:44:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1085469684 shutdown_connections 2024-09-13T23:44:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1085469684 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1085469684 >> 172.21.15.167:0/1085469684 conn(0x55966f8a4c00 msgr2=0x55966f9a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1085469684 shutdown_connections 2024-09-13T23:44:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1085469684 wait complete. 2024-09-13T23:44:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 Processor -- start 2024-09-13T23:44:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 -- start start 2024-09-13T23:44:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb4d73f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966f9323c0 con 0x55966f8a5000 2024-09-13T23:44:35.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb352c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb352c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43054/0 (socket says 172.21.15.167:43054) 2024-09-13T23:44:35.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.865+0000 7f8eb352c700 1 -- 172.21.15.167:0/817330095 learned_addr learned my addr 172.21.15.167:0/817330095 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb352c700 1 -- 172.21.15.167:0/817330095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55966f8b7a00 con 0x55966f8a5000 2024-09-13T23:44:35.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb352c700 1 --2- 172.21.15.167:0/817330095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x559670709f20 tx=0x55966f9a9920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb0526700 1 -- 172.21.15.167:0/817330095 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966f949600 con 0x55966f8a5000 2024-09-13T23:44:35.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/817330095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55966f8b7860 con 0x55966f8a5000 2024-09-13T23:44:35.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb0526700 1 -- 172.21.15.167:0/817330095 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55966f969c80 con 0x55966f8a5000 2024-09-13T23:44:35.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb0526700 1 -- 172.21.15.167:0/817330095 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966f9498c0 con 0x55966f8a5000 2024-09-13T23:44:35.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/817330095 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55966f8b76c0 con 0x55966f8a5000 2024-09-13T23:44:35.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb4d73f40 1 Processor -- start 2024-09-13T23:44:35.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb4d73f40 1 -- start start 2024-09-13T23:44:35.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb4d73f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 0x55966fb8ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb4d73f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966f8ca640 con 0x55966f8a5800 2024-09-13T23:44:35.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb3d2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 0x55966fb8ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb3d2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 0x55966fb8ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43056/0 (socket says 172.21.15.167:43056) 2024-09-13T23:44:35.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/661928557 learned_addr learned my addr 172.21.15.167:0/661928557 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb0526700 1 -- 172.21.15.167:0/817330095 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55966f99eb00 con 0x55966f8a5000 2024-09-13T23:44:35.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/661928557 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55966f8b6000 con 0x55966f8a5800 2024-09-13T23:44:35.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb0526700 1 --2- 172.21.15.167:0/817330095 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fc54000 0x55966fb8b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb0526700 1 -- 172.21.15.167:0/817330095 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55966f90b180 con 0x55966f8a5000 2024-09-13T23:44:35.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb352c700 1 --2- 172.21.15.167:0/817330095 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fc54000 0x55966fb8b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eb3d2d700 1 --2- 172.21.15.167:0/661928557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 0x55966fb8ab00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x559670718f00 tx=0x55966fc088a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614adceb6eb1c8f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eadd21700 1 -- 172.21.15.167:0/661928557 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966f949600 con 0x55966f8a5800 2024-09-13T23:44:35.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.869+0000 7f8eadd21700 1 -- 172.21.15.167:0/661928557 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55966fc16000 con 0x55966f8a5800 2024-09-13T23:44:35.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eadd21700 1 -- 172.21.15.167:0/661928557 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966f9498c0 con 0x55966f8a5800 2024-09-13T23:44:35.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb352c700 1 --2- 172.21.15.167:0/817330095 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fc54000 0x55966fb8b080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5596707190e0 tx=0x55966fc08120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/661928557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 msgr2=0x55966fb8ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/661928557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 0x55966fb8ab00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x559670718f00 tx=0x55966fc088a0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/661928557 shutdown_connections 2024-09-13T23:44:35.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/661928557 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 0x55966fb8ab00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/661928557 >> 172.21.15.167:0/661928557 conn(0x55966f8a5400 msgr2=0x55966f9a5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/661928557 shutdown_connections 2024-09-13T23:44:35.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/661928557 wait complete. 2024-09-13T23:44:35.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb4d73f40 1 Processor -- start 2024-09-13T23:44:35.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb4d73f40 1 -- start start 2024-09-13T23:44:35.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb4d73f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 0x55966fb8ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb4d73f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966f8ca8c0 con 0x55966f8a5800 2024-09-13T23:44:35.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb3d2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 0x55966fb8ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb3d2d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 0x55966fb8ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43070/0 (socket says 172.21.15.167:43070) 2024-09-13T23:44:35.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 learned_addr learned my addr 172.21.15.167:0/55888693 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.873+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55966f8b6ea0 con 0x55966f8a5800 2024-09-13T23:44:35.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eb3d2d700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 0x55966fb8ab00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x5596707182d0 tx=0x55966f9a9b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eac51e700 1 -- 172.21.15.167:0/55888693 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966f949760 con 0x55966f8a5800 2024-09-13T23:44:35.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eac51e700 1 -- 172.21.15.167:0/55888693 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55966fc16180 con 0x55966f8a5800 2024-09-13T23:44:35.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eac51e700 1 -- 172.21.15.167:0/55888693 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966f949600 con 0x55966f8a5800 2024-09-13T23:44:35.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55966f8b6820 con 0x55966f8a5800 2024-09-13T23:44:35.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55966f8b69c0 con 0x55966f8a5800 2024-09-13T23:44:35.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eac51e700 1 -- 172.21.15.167:0/55888693 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55966f99edc0 con 0x55966f8a5800 2024-09-13T23:44:35.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eac51e700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fc54c00 0x55966fb8b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eac51e700 1 -- 172.21.15.167:0/55888693 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55966f90b340 con 0x55966f8a5800 2024-09-13T23:44:35.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eb2d2b700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fc54c00 0x55966fb8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55966fc55000 0x55966fb8bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55400 con 0x55966fc55000 2024-09-13T23:44:35.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.877+0000 7f8eb3d2d700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55966fc55000 0x55966fb8bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb2d2b700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fc54c00 0x55966fb8b600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55966fc53b30 tx=0x55966fc09f80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb3d2d700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55966fc55000 0x55966fb8bb80 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 20 realm 2024-09-13T23:44:35.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55000 2024-09-13T23:44:35.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:35.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55c00 con 0x55966fc55000 2024-09-13T23:44:35.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55966fc55400 0x55966fb8c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55400 2024-09-13T23:44:35.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb352c700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55966fc55400 0x55966fb8c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb352c700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55966fc55400 0x55966fb8c100 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.881+0000 7f8eb4d73f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:35.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55c00 con 0x55966fc55400 2024-09-13T23:44:35.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb4d73f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:35.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb4d73f40 20 searching for the correct realm 2024-09-13T23:44:35.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55966fd10000 0x55966fb8c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10400 con 0x55966fd10000 2024-09-13T23:44:35.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb2d2b700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55966fd10000 0x55966fb8c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb2d2b700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55966fd10000 0x55966fb8c680 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10400 con 0x55966fc55400 2024-09-13T23:44:35.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb352c700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55966fd10400 0x55966fb8cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10400 2024-09-13T23:44:35.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb3d2d700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55966fd10400 0x55966fb8cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.885+0000 7f8eb3d2d700 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55966fd10400 0x55966fb8cc00 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:35.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55000 2024-09-13T23:44:35.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55000 2024-09-13T23:44:35.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10000 2024-09-13T23:44:35.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55000 2024-09-13T23:44:35.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55400 2024-09-13T23:44:35.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55400 2024-09-13T23:44:35.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.889+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55000 2024-09-13T23:44:35.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10400 2024-09-13T23:44:35.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:35.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55000 2024-09-13T23:44:35.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10400 2024-09-13T23:44:35.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:35.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55400 2024-09-13T23:44:35.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10000 2024-09-13T23:44:35.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55000 2024-09-13T23:44:35.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.893+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55400 2024-09-13T23:44:35.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10000 2024-09-13T23:44:35.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55000 2024-09-13T23:44:35.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10400 2024-09-13T23:44:35.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:35.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55000 2024-09-13T23:44:35.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10000 2024-09-13T23:44:35.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10400 2024-09-13T23:44:35.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:35.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55000 2024-09-13T23:44:35.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.897+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10400 2024-09-13T23:44:35.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.901+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:35.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.901+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10000 2024-09-13T23:44:35.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.901+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.901+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10400 2024-09-13T23:44:35.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.901+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:35.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.901+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55400 2024-09-13T23:44:35.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.901+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.901+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55400 2024-09-13T23:44:35.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.901+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.907 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.901+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10000 2024-09-13T23:44:35.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.901+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.901+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55400 2024-09-13T23:44:35.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fc55400 2024-09-13T23:44:35.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:35.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:35.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:35.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:35.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10800 con 0x55966fc55000 2024-09-13T23:44:35.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.911 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10c00 con 0x55966fc55400 2024-09-13T23:44:35.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:35.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.912 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11000 con 0x55966fc55400 2024-09-13T23:44:35.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.905+0000 7f8eb4d73f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:35.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 20 zone default found 2024-09-13T23:44:35.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 4 Realm: () 2024-09-13T23:44:35.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:35.914 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 20 zonegroup default 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11400 con 0x55966fd10400 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11800 con 0x55966fd10000 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11c00 con 0x55966fd10000 2024-09-13T23:44:35.915 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:35.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 20 started sync module instance, tier type = 2024-09-13T23:44:35.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:35.916 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.909+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11800 con 0x55966fc55400 2024-09-13T23:44:35.917 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.913+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.913+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94104607791872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11c00 con 0x55966fc55400 2024-09-13T23:44:35.918 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.913+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10c00 con 0x55966fc55000 2024-09-13T23:44:35.920 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.917+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.917+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94104607793024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11000 con 0x55966fc55000 2024-09-13T23:44:35.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.917+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11800 con 0x55966fc55400 2024-09-13T23:44:35.921 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.917+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94104607791872] v35'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.917+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94104607793024] v35'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.917+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.917+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94104607794176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55c00 con 0x55966fc55400 2024-09-13T23:44:35.923 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.917+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55800 con 0x55966fd10400 2024-09-13T23:44:35.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.921+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:35.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.921+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94104607795328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10800 con 0x55966fd10400 2024-09-13T23:44:35.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.921+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11400 con 0x55966fc55400 2024-09-13T23:44:35.926 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.921+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94104607794176] v35'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.921+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.921+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94104607796480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10c00 con 0x55966fc55400 2024-09-13T23:44:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.921+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11000 con 0x55966fd10000 2024-09-13T23:44:35.928 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.925+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94104607795328] v35'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:35.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.925+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94104607796480] v35'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:35.930 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.925+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.925+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94104607797632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd12000 con 0x55966fd10000 2024-09-13T23:44:35.931 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.925+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd12400 con 0x55966fd10000 2024-09-13T23:44:35.932 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94104607797632] v35'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94104607798784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd12800 con 0x55966fd10000 2024-09-13T23:44:35.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd12c00 con 0x55966fd10400 2024-09-13T23:44:35.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'222 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:35.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94104607799936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13000 con 0x55966fd10400 2024-09-13T23:44:35.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb4d73f40 20 add_watcher() i=0 2024-09-13T23:44:35.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb4d73f40 20 add_watcher() i=1 2024-09-13T23:44:35.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb4d73f40 20 add_watcher() i=2 2024-09-13T23:44:35.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb4d73f40 20 add_watcher() i=3 2024-09-13T23:44:35.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb4d73f40 20 add_watcher() i=4 2024-09-13T23:44:35.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb4d73f40 20 add_watcher() i=5 2024-09-13T23:44:35.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94104607798784] v35'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:35.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.929+0000 7f8eb4d73f40 20 add_watcher() i=6 2024-09-13T23:44:35.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.933+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94104607799936] v35'223 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:35.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.933+0000 7f8eb4d73f40 20 add_watcher() i=7 2024-09-13T23:44:35.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.933+0000 7f8eb4d73f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:35.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.933+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13400 con 0x55966fc55000 2024-09-13T23:44:35.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.933+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.933+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94104609456128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13800 con 0x55966fc55000 2024-09-13T23:44:35.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.933+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94104609456128] v35'218 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:35.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.933+0000 7f8e9a4fa700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:35.939 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.933+0000 7f8eb4d73f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:35.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.933+0000 7f8eb4d73f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:35.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.933+0000 7f8eb4d73f40 1 Processor -- start 2024-09-13T23:44:35.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.933+0000 7f8eb4d73f40 1 -- start start 2024-09-13T23:44:35.940 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 0x55966fb8d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966f9323c0 con 0x55966fd12c00 2024-09-13T23:44:35.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb2d2b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 0x55966fb8d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb2d2b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 0x55966fb8d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43076/0 (socket says 172.21.15.167:43076) 2024-09-13T23:44:35.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/2273015042 learned_addr learned my addr 172.21.15.167:0/2273015042 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/2273015042 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55966f8b6b60 con 0x55966fd12c00 2024-09-13T23:44:35.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb2d2b700 1 --2- 172.21.15.167:0/2273015042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 0x55966fb8d180 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x55966fd22090 tx=0x55966fd24000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dd09019582c84d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8e99cf9700 1 -- 172.21.15.167:0/2273015042 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966fd2c000 con 0x55966fd12c00 2024-09-13T23:44:35.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8e99cf9700 1 -- 172.21.15.167:0/2273015042 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55966fc16600 con 0x55966fd12c00 2024-09-13T23:44:35.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8e99cf9700 1 -- 172.21.15.167:0/2273015042 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966fd2c160 con 0x55966fd12c00 2024-09-13T23:44:35.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/2273015042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 msgr2=0x55966fb8d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:35.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/2273015042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 0x55966fb8d180 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x55966fd22090 tx=0x55966fd24000 comp rx=0 tx=0).stop 2024-09-13T23:44:35.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/2273015042 shutdown_connections 2024-09-13T23:44:35.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/2273015042 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 0x55966fb8d180 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:35.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/2273015042 >> 172.21.15.167:0/2273015042 conn(0x55966fd13c00 msgr2=0x55966f9a6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:35.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/2273015042 shutdown_connections 2024-09-13T23:44:35.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/2273015042 wait complete. 2024-09-13T23:44:35.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 Processor -- start 2024-09-13T23:44:35.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 -- start start 2024-09-13T23:44:35.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 0x55966fb8d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.937+0000 7f8eb4d73f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55966f932500 con 0x55966fd12c00 2024-09-13T23:44:35.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8eb2d2b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 0x55966fb8d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8eb2d2b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 0x55966fb8d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43078/0 (socket says 172.21.15.167:43078) 2024-09-13T23:44:35.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/1927905630 learned_addr learned my addr 172.21.15.167:0/1927905630 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:35.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55966f8b6d00 con 0x55966fd12c00 2024-09-13T23:44:35.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8eb2d2b700 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 0x55966fb8d180 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x55966fd23350 tx=0x55966fd242a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8e984f6700 1 -- 172.21.15.167:0/1927905630 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966fd2c2c0 con 0x55966fd12c00 2024-09-13T23:44:35.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8e984f6700 1 -- 172.21.15.167:0/1927905630 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55966fc16300 con 0x55966fd12c00 2024-09-13T23:44:35.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55966f8b6340 con 0x55966fd12c00 2024-09-13T23:44:35.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8e984f6700 1 -- 172.21.15.167:0/1927905630 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55966fd2c000 con 0x55966fd12c00 2024-09-13T23:44:35.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55966f8b6b60 con 0x55966fd12c00 2024-09-13T23:44:35.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8e984f6700 1 -- 172.21.15.167:0/1927905630 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55966f99fb80 con 0x55966fd12c00 2024-09-13T23:44:35.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8e984f6700 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fda6000 0x55966fdac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8e984f6700 1 -- 172.21.15.167:0/1927905630 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55966f90b500 con 0x55966fd12c00 2024-09-13T23:44:35.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8eb3d2d700 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fda6000 0x55966fdac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55966fd13000 0x55966fb8d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.941+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13400 con 0x55966fd13000 2024-09-13T23:44:35.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.945+0000 7f8eb352c700 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55966fd13000 0x55966fb8d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.945+0000 7f8eb3d2d700 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fda6000 0x55966fdac000 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55966fc52e70 tx=0x55966fc093e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.945+0000 7f8eb352c700 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55966fd13000 0x55966fb8d700 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.945+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v35'38 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:35.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.945+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55966fd12000 0x55966fdac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.945+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd12400 con 0x55966fd12000 2024-09-13T23:44:35.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.945+0000 7f8eb2d2b700 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55966fd12000 0x55966fdac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.949+0000 7f8eb2d2b700 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55966fd12000 0x55966fdac580 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.949+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/1927905630 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v35'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd12000 2024-09-13T23:44:35.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.949+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55966fd10800 0x55966fdacb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.949+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11000 con 0x55966fd10800 2024-09-13T23:44:35.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.949+0000 7f8eb3d2d700 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55966fd10800 0x55966fdacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.955 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.949+0000 7f8eb3d2d700 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55966fd10800 0x55966fdacb00 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.953+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/1927905630 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v35'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10800 2024-09-13T23:44:35.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.953+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55966fc55800 0x55966fdad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:35.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.953+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6400 con 0x55966fc55800 2024-09-13T23:44:35.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.953+0000 7f8eb352c700 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55966fc55800 0x55966fdad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:35.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.953+0000 7f8eb352c700 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55966fc55800 0x55966fdad080 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:35.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.957+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v35'219 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55800 2024-09-13T23:44:35.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.957+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6800 con 0x55966fd13000 2024-09-13T23:44:35.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.957+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v35'68 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:35.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.957+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6c00 con 0x55966fd10800 2024-09-13T23:44:35.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.961+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/1927905630 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v35'48 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10800 2024-09-13T23:44:35.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.961+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7000 con 0x55966fd13000 2024-09-13T23:44:35.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.965+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v35'28 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:35.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.965+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7400 con 0x55966fc55800 2024-09-13T23:44:35.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.965+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v35'46 uv35 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55800 2024-09-13T23:44:35.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.965+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7800 con 0x55966fd13000 2024-09-13T23:44:35.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.969+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v35'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:35.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.969+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7c00 con 0x55966fd13000 2024-09-13T23:44:35.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.973+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v35'33 uv28 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:35.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.973+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13400 con 0x55966fd13000 2024-09-13T23:44:35.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.973+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v35'69 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:35.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.973+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11c00 con 0x55966fd10800 2024-09-13T23:44:35.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.977+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/1927905630 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v35'30 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10800 2024-09-13T23:44:35.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.977+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11800 con 0x55966fd13000 2024-09-13T23:44:35.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.981+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v35'29 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:35.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.981+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11400 con 0x55966fc55800 2024-09-13T23:44:35.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.981+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v35'51 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55800 2024-09-13T23:44:35.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.981+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10c00 con 0x55966fc55800 2024-09-13T23:44:35.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.985+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v35'220 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55800 2024-09-13T23:44:35.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.985+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55c00 con 0x55966fd13000 2024-09-13T23:44:35.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.989+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v35'65 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:35.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.989+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13400 con 0x55966fd13000 2024-09-13T23:44:35.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.989+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v35'70 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:35.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.993+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6400 con 0x55966fd10800 2024-09-13T23:44:35.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.993+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/1927905630 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v35'28 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10800 2024-09-13T23:44:35.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.993+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6800 con 0x55966fd13000 2024-09-13T23:44:36.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.997+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v35'35 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:36.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.997+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7000 con 0x55966fc55800 2024-09-13T23:44:36.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.997+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v35'221 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55800 2024-09-13T23:44:36.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:35.997+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7400 con 0x55966fd13000 2024-09-13T23:44:36.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.001+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v35'66 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:36.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.001+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7800 con 0x55966fd13000 2024-09-13T23:44:36.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.005+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v35'36 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:36.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.005+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7c00 con 0x55966fd10800 2024-09-13T23:44:36.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.005+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/1927905630 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v35'28 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10800 2024-09-13T23:44:36.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.005+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6c00 con 0x55966fd13000 2024-09-13T23:44:36.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.009+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v35'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:36.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.009+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6400 con 0x55966fd10800 2024-09-13T23:44:36.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.013+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/1927905630 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v35'22 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10800 2024-09-13T23:44:36.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.013+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7c00 con 0x55966fd10800 2024-09-13T23:44:36.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.017+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/1927905630 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v35'49 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10800 2024-09-13T23:44:36.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.017+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6800 con 0x55966fd10800 2024-09-13T23:44:36.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.017+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/1927905630 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v35'28 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10800 2024-09-13T23:44:36.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.017+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7000 con 0x55966fd13000 2024-09-13T23:44:36.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.021+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v35'26 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:36.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.021+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7400 con 0x55966fc55800 2024-09-13T23:44:36.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.025+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v35'47 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55800 2024-09-13T23:44:36.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.025+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13800 con 0x55966fd13000 2024-09-13T23:44:36.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.029+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v35'67 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fd13000 2024-09-13T23:44:36.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.029+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13400 con 0x55966fc55800 2024-09-13T23:44:36.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.029+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v35'52 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55800 2024-09-13T23:44:36.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.029+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55c00 con 0x55966fd12000 2024-09-13T23:44:36.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.033+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/1927905630 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v35'32 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd12000 2024-09-13T23:44:36.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8e95cf1700 20 reqs_thread_entry: start 2024-09-13T23:44:36.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:36.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:36.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8e934ec700 20 reqs_thread_entry: start 2024-09-13T23:44:36.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-09-13T23:44:36.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11800 con 0x55966fd10400 2024-09-13T23:44:36.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8e93ced700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:45:06 2024 2024-09-13T23:44:36.042 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:36.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(62 bud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv2 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 20 rados_obj.operate() r=0 bl.length=346 2024-09-13T23:44:36.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-09-13T23:44:36.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-09-13T23:44:36.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:44:36.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:63 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11400 con 0x55966fc55400 2024-09-13T23:44:36.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(63 bud.buckets [call out=184b] v0'0 uv3 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 : miss 2024-09-13T23:44:36.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.037+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 8.1a 8:5c084470:root::.bucket.meta.myfoo6%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10c00 con 0x55966fd10000 2024-09-13T23:44:36.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(64 .bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 [call out=48b,stat out=16b,read 0~353 out=353b,getxattrs out=212b] v0'0 uv4 ondisk = 0) v8 ==== 333+0+629 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb4d73f40 20 rados_obj.operate() r=0 bl.length=353 2024-09-13T23:44:36.045 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb4d73f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 info.flags=0x17 2024-09-13T23:44:36.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb4d73f40 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 to cache LRU end 2024-09-13T23:44:36.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb4d73f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:44:36.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb4d73f40 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-09-13T23:44:36.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb4d73f40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 2024-09-13T23:44:36.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 4.1a 4:5a627cc1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed8000 con 0x55966fc55400 2024-09-13T23:44:36.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.0 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 4.12 4:4bdfb1b9:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed8400 con 0x55966fc55000 2024-09-13T23:44:36.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(66 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 4.36 4:6f4c2246:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed8800 con 0x55966fc55000 2024-09-13T23:44:36.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(67 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.2 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 4.c 4:3312ce25:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed8c00 con 0x55966fc55000 2024-09-13T23:44:36.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(68 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.3 [call out=85b] v0'0 uv11 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.041+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 4.10 4:09dc872b:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9000 con 0x55966fd10000 2024-09-13T23:44:36.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.045+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(69 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.045+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 4.12 4:499b0777:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9400 con 0x55966fc55000 2024-09-13T23:44:36.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.045+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(70 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.5 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.045+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 4.3e 4:7ead4b73:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9800 con 0x55966fc55400 2024-09-13T23:44:36.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.045+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(71 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.6 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.045+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 4.22 4:44677d15:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9c00 con 0x55966fc55000 2024-09-13T23:44:36.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.045+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(72 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.045+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 4.5 4:a040ca26:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed8000 con 0x55966fc55400 2024-09-13T23:44:36.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.045+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(73 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.045+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 4.34 4:2f86a938:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9800 con 0x55966fd10400 2024-09-13T23:44:36.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(74 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9 [call out=289b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+289 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb4d73f40 10 cls_bucket_list_unordered: got thirteen 2024-09-13T23:44:36.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 4.31 4:8f1da0e5:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9000 con 0x55966fc55400 2024-09-13T23:44:36.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(75 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb4d73f40 20 get_obj_state: rctx=0x55966f935730 obj=myfoo6:thirteen state=0x55966fe5c1e8 s->prefetch_data=0 2024-09-13T23:44:36.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 3.27s0 3:e602001c:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11400 con 0x55966fd10000 2024-09-13T23:44:36.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(76 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7_thirteen [getxattrs out=819b,stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 238+0+835 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb4d73f40 10 manifest: total_size = 8 2024-09-13T23:44:36.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb4d73f40 20 get_obj_state: setting s->obj_tag to _O7rVFYbJUqlVdSbKMfA0AEOSZ-hYaKf 2024-09-13T23:44:36.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb4d73f40 20 get_obj_state: rctx=0x55966f935730 obj=myfoo6:thirteen state=0x55966fe5c1e8 s->prefetch_data=0 2024-09-13T23:44:36.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb4d73f40 20 prepare_atomic_modification: state is not atomic. state=0x55966fe5c1e8 2024-09-13T23:44:36.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb4d73f40 20 bucket index object: default.rgw.buckets.index:.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9 2024-09-13T23:44:36.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.049+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:77 4.34 4:2f86a938:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10c00 con 0x55966fd10400 2024-09-13T23:44:36.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.053+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(77 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9 [stat,call,call] v35'6 uv6 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.053+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:78 3.27s0 3:e602001c:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7_thirteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11800 con 0x55966fd10000 2024-09-13T23:44:36.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(78 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7_thirteen [call] v35'6 uv6 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:79 4.34 4:2f86a938:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=264b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6c00 con 0x55966fd10400 2024-09-13T23:44:36.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 20 list_objects_ordered: starting attempt 1 2024-09-13T23:44:36.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-09-13T23:44:36.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 4.1a 4:5a627cc1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7000 con 0x55966fc55400 2024-09-13T23:44:36.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:81 4.12 4:4bdfb1b9:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7400 con 0x55966fc55000 2024-09-13T23:44:36.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:82 4.36 4:6f4c2246:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6400 con 0x55966fc55000 2024-09-13T23:44:36.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:83 4.c 4:3312ce25:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7c00 con 0x55966fc55000 2024-09-13T23:44:36.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:84 4.10 4:09dc872b:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6800 con 0x55966fd10000 2024-09-13T23:44:36.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:85 4.12 4:499b0777:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55c00 con 0x55966fc55000 2024-09-13T23:44:36.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:86 4.3e 4:7ead4b73:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13800 con 0x55966fc55400 2024-09-13T23:44:36.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:87 4.22 4:44677d15:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13400 con 0x55966fc55000 2024-09-13T23:44:36.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:88 4.5 4:a040ca26:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd12400 con 0x55966fc55400 2024-09-13T23:44:36.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:89 4.34 4:2f86a938:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9400 con 0x55966fd10400 2024-09-13T23:44:36.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:90 4.31 4:8f1da0e5:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9c00 con 0x55966fc55400 2024-09-13T23:44:36.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(84 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(81 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(82 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.2 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(85 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.5 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(83 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.3 [call out=85b] v0'0 uv11 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(80 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.0 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.057+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(87 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(79 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9 [stat,call,call] v35'7 uv7 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eadd21700 20 handle_completion(): completion ok for obj=thirteen 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(86 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.6 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(90 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(88 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(89 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-09-13T23:44:36.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-09-13T23:44:36.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo6[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7])"; uploads.size()=0; is_truncated=0 2024-09-13T23:44:36.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo6[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7]) 2024-09-13T23:44:36.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo6 : miss 2024-09-13T23:44:36.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 20 lifecycle: rados->read ofs=0 len=0 2024-09-13T23:44:36.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:91 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9800 con 0x55966fc55000 2024-09-13T23:44:36.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 28 ==== osd_op_reply(91 myfoo6 [call out=48b,stat out=16b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv18 ondisk = 0) v8 ==== 276+0+305 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 20 lifecycle: rados_obj.operate() r=0 bl.length=166 2024-09-13T23:44:36.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x17 2024-09-13T23:44:36.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 10 lifecycle: adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-09-13T23:44:36.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:44:36.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.061+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:92 8.1a 8:5c084470:root::.bucket.meta.myfoo6%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed8800 con 0x55966fd10000 2024-09-13T23:44:36.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(92 .bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 [delete,create,call,call,writefull 0~353,setxattr (117)] v35'5 uv5 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb4d73f40 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 info.flags=0x17 2024-09-13T23:44:36.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb4d73f40 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 to cache LRU end 2024-09-13T23:44:36.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb4d73f40 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=117 2024-09-13T23:44:36.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb4d73f40 10 lifecycle: distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7, ofs0, ns] 2024-09-13T23:44:36.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:93 7.e 7:7759931f:::notify.3:head [notify cookie 94104609459584 in=733b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6c00 con 0x55966fd10400 2024-09-13T23:44:36.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 18 ==== watch-notify(notify (1) cookie 94104607795328 notify 150323855390 ret 0) v3 ==== 763+0+0 (crc 0 0 0) 0x55966fcb3380 con 0x55966fd10400 2024-09-13T23:44:36.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8ead520700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855390 cookie 94104607795328 notifier 4822 bl.length()=721 2024-09-13T23:44:36.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 19 ==== osd_op_reply(93 notify.3 [notify cookie 94104609459584 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8ead520700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 info.flags=0x17 2024-09-13T23:44:36.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8ead520700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 to cache LRU end 2024-09-13T23:44:36.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8ead520700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-09-13T23:44:36.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8ead520700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:94 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe84000 con 0x55966fd10400 2024-09-13T23:44:36.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 20 ==== osd_op_reply(94 notify.3 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 21 ==== watch-notify(notify_complete (2) cookie 94104609459584 notify 150323855390 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55966fcb3520 con 0x55966fd10400 2024-09-13T23:44:36.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb4d73f40 20 lifecycle: handle_bi_update(): bucket=:myfoo6[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7]): orig_sources= new_sources= 2024-09-13T23:44:36.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb4d73f40 20 lifecycle: handle_bi_update(): bucket=:myfoo6[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7]): potential sources added=[] removed=[] 2024-09-13T23:44:36.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb4d73f40 20 lifecycle: handle_bi_update(): bucket=:myfoo6[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7]): orig_dests= new_dests= 2024-09-13T23:44:36.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb4d73f40 20 lifecycle: handle_bi_update(): bucket=:myfoo6[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7]): potential dests added=[] removed=[] 2024-09-13T23:44:36.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.065+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:33 6.14 6:2e7dc4d8:lc::lc.22:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7400 con 0x55966fd12000 2024-09-13T23:44:36.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.069+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/1927905630 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(33 lc.22 [call] v35'11 uv11 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd12000 2024-09-13T23:44:36.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.069+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:34 6.14 6:2e7dc4d8:lc::lc.22:head [call rgw.lc_rm_entry in=93b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6400 con 0x55966fd12000 2024-09-13T23:44:36.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.069+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/1927905630 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(34 lc.22 [call] v35'12 uv12 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd12000 2024-09-13T23:44:36.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.069+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:35 6.14 6:2e7dc4d8:lc::lc.22:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7c00 con 0x55966fd12000 2024-09-13T23:44:36.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/1927905630 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(35 lc.22 [call] v35'13 uv13 ondisk = 0) v8 ==== 149+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd12000 2024-09-13T23:44:36.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:95 4.1a 4:5a627cc1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10c00 con 0x55966fc55400 2024-09-13T23:44:36.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:96 4.12 4:4bdfb1b9:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe84400 con 0x55966fc55000 2024-09-13T23:44:36.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:97 4.36 4:6f4c2246:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe84800 con 0x55966fc55000 2024-09-13T23:44:36.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:98 4.c 4:3312ce25:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe84c00 con 0x55966fc55000 2024-09-13T23:44:36.079 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:99 4.10 4:09dc872b:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe85000 con 0x55966fd10000 2024-09-13T23:44:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:100 4.12 4:499b0777:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe85400 con 0x55966fc55000 2024-09-13T23:44:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:101 4.3e 4:7ead4b73:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe85800 con 0x55966fc55400 2024-09-13T23:44:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:102 4.22 4:44677d15:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe85c00 con 0x55966fc55000 2024-09-13T23:44:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:103 4.5 4:a040ca26:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6800 con 0x55966fc55400 2024-09-13T23:44:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:104 4.34 4:2f86a938:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7400 con 0x55966fd10400 2024-09-13T23:44:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:105 4.31 4:8f1da0e5:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed8800 con 0x55966fc55400 2024-09-13T23:44:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(99 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 29 ==== osd_op_reply(96 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 30 ==== osd_op_reply(97 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.2 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 31 ==== osd_op_reply(100 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.5 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 32 ==== osd_op_reply(98 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.3 [call out=85b] v0'0 uv11 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 33 ==== osd_op_reply(102 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.7 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(95 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.0 [call out=85b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(101 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.6 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 31 ==== osd_op_reply(103 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.8 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 32 ==== osd_op_reply(105 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 22 ==== osd_op_reply(104 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9 [call out=124b] v0'0 uv7 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.073+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:106 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed8400 con 0x55966fc55400 2024-09-13T23:44:36.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.077+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 33 ==== osd_op_reply(106 bud.buckets [call] v35'4 uv4 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.077+0000 7f8eb4d73f40 10 cache get: name=default.rgw.meta+root+myfoo6 : hit (requested=0x11, cached=0x17) 2024-09-13T23:44:36.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.077+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:107 8.18 8:1944a257:root::myfoo6:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed8c00 con 0x55966fc55000 2024-09-13T23:44:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.077+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 34 ==== osd_op_reply(107 myfoo6 [call,call,delete] v35'19 uv19 ondisk = 0) v8 ==== 234+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.077+0000 7f8eb4d73f40 10 removing default.rgw.meta+root+myfoo6 from cache 2024-09-13T23:44:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.077+0000 7f8eb4d73f40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-09-13T23:44:36.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.077+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [notify cookie 94104609459584 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11800 con 0x55966fd10400 2024-09-13T23:44:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.077+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 23 ==== watch-notify(notify (1) cookie 94104607795328 notify 150323855391 ret 0) v3 ==== 192+0+0 (crc 0 0 0) 0x55966fcb3520 con 0x55966fd10400 2024-09-13T23:44:36.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.077+0000 7f8ead520700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855391 cookie 94104607795328 notifier 4822 bl.length()=150 2024-09-13T23:44:36.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.077+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 24 ==== osd_op_reply(108 notify.3 [notify cookie 94104609459584 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.077+0000 7f8ead520700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55c00 con 0x55966fd10400 2024-09-13T23:44:36.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 25 ==== osd_op_reply(109 notify.3 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 26 ==== watch-notify(notify_complete (2) cookie 94104609459584 notify 150323855391 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55966fcb36c0 con 0x55966fd10400 2024-09-13T23:44:36.085 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:110 8.1a 8:5c084470:root::.bucket.meta.myfoo6%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11400 con 0x55966fd10000 2024-09-13T23:44:36.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(110 .bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 [call,call,delete] v35'6 uv6 ondisk = 0) v8 ==== 291+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 10 removing default.rgw.meta+root+.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 from cache 2024-09-13T23:44:36.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo6:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7, ofs0, ns] 2024-09-13T23:44:36.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:111 7.e 7:7759931f:::notify.3:head [notify cookie 94104609459584 in=219b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe85800 con 0x55966fd10400 2024-09-13T23:44:36.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 27 ==== watch-notify(notify (1) cookie 94104607795328 notify 150323855392 ret 0) v3 ==== 249+0+0 (crc 0 0 0) 0x55966fcb36c0 con 0x55966fd10400 2024-09-13T23:44:36.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8ead520700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855392 cookie 94104607795328 notifier 4822 bl.length()=207 2024-09-13T23:44:36.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 28 ==== osd_op_reply(111 notify.3 [notify cookie 94104609459584 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8ead520700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:112 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10c00 con 0x55966fd10400 2024-09-13T23:44:36.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 29 ==== watch-notify(notify_complete (2) cookie 94104609459584 notify 150323855392 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55966fcb3860 con 0x55966fd10400 2024-09-13T23:44:36.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 30 ==== osd_op_reply(112 notify.3 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:113 4.1a 4:5a627cc1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe84000 con 0x55966fc55400 2024-09-13T23:44:36.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:114 4.12 4:4bdfb1b9:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe84400 con 0x55966fc55000 2024-09-13T23:44:36.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:115 4.36 4:6f4c2246:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe84800 con 0x55966fc55000 2024-09-13T23:44:36.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:116 4.c 4:3312ce25:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe84c00 con 0x55966fc55000 2024-09-13T23:44:36.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:117 4.10 4:09dc872b:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7000 con 0x55966fd10000 2024-09-13T23:44:36.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:118 4.12 4:499b0777:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6800 con 0x55966fc55000 2024-09-13T23:44:36.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:119 4.3e 4:7ead4b73:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6c00 con 0x55966fc55400 2024-09-13T23:44:36.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:120 4.22 4:44677d15:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed8000 con 0x55966fc55000 2024-09-13T23:44:36.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:121 4.5 4:a040ca26:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9000 con 0x55966fc55400 2024-09-13T23:44:36.089 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:122 4.34 4:2f86a938:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9c00 con 0x55966fd10400 2024-09-13T23:44:36.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.081+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:123 4.31 4:8f1da0e5:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed8800 con 0x55966fc55400 2024-09-13T23:44:36.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.085+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 34 ==== osd_op_reply(113 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.0 [delete] v35'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.085+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(117 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.4 [delete] v35'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.085+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 35 ==== osd_op_reply(116 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.3 [delete] v35'12 uv12 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.085+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 36 ==== osd_op_reply(115 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.2 [delete] v35'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.091 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.085+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 31 ==== osd_op_reply(122 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.9 [delete] v35'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.085+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 35 ==== osd_op_reply(119 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.6 [delete] v35'5 uv5 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.085+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 37 ==== osd_op_reply(114 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.1 [delete] v35'3 uv3 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.085+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 38 ==== osd_op_reply(120 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.7 [delete] v35'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.085+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 36 ==== osd_op_reply(121 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.8 [delete] v35'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.085+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 37 ==== osd_op_reply(123 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.10 [delete] v35'2 uv2 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.089+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 39 ==== osd_op_reply(118 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7.5 [delete] v35'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.089+0000 7f8eb4d73f40 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo6/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 : miss 2024-09-13T23:44:36.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.089+0000 7f8eb4d73f40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.089+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:124 6.0 6:05ffc37a:::pubsub..bucket.myfoo6%2f9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9400 con 0x55966fc55000 2024-09-13T23:44:36.093 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.089+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 40 ==== osd_op_reply(124 pubsub..bucket.myfoo6/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.089+0000 7f8eb4d73f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:36.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.089+0000 7f8eb4d73f40 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo6/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 info.flags=0x0 2024-09-13T23:44:36.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.089+0000 7f8eb4d73f40 10 adding default.rgw.log++pubsub..bucket.myfoo6/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 to cache LRU end 2024-09-13T23:44:36.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.089+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:125 6.0 6:05ffc37a:::pubsub..bucket.myfoo6%2f9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fed9800 con 0x55966fc55000 2024-09-13T23:44:36.094 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.089+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 41 ==== osd_op_reply(125 pubsub..bucket.myfoo6/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.7 [delete] v35'53 uv42 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.089+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:126 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13800 con 0x55966fc55400 2024-09-13T23:44:36.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 38 ==== osd_op_reply(126 bud.buckets [call] v35'5 uv5 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb4d73f40 10 removing key index: NX5QOQKC6BH2IDN8HC7A 2024-09-13T23:44:36.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:127 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd12400 con 0x55966fc55000 2024-09-13T23:44:36.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 42 ==== osd_op_reply(127 NX5QOQKC6BH2IDN8HC7A [delete] v35'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb4d73f40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-09-13T23:44:36.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:128 7.3 7:c609908c:::notify.5:head [notify cookie 94104609459584 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13400 con 0x55966fd10000 2024-09-13T23:44:36.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== watch-notify(notify (1) cookie 94104607797632 notify 150323855379 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x55966fd70b60 con 0x55966fd10000 2024-09-13T23:44:36.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8ead520700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855379 cookie 94104607797632 notifier 4822 bl.length()=170 2024-09-13T23:44:36.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== osd_op_reply(128 notify.5 [notify cookie 94104609459584 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8ead520700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:129 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55c00 con 0x55966fd10000 2024-09-13T23:44:36.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 25 ==== watch-notify(notify_complete (2) cookie 94104609459584 notify 150323855379 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55966fd70d00 con 0x55966fd10000 2024-09-13T23:44:36.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb4d73f40 10 removing email index: 2024-09-13T23:44:36.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb4d73f40 10 removing user buckets index 2024-09-13T23:44:36.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 26 ==== osd_op_reply(129 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.093+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:130 8.19 8:99adfd5d:users.uid::bud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe85400 con 0x55966fc55400 2024-09-13T23:44:36.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.097+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 39 ==== osd_op_reply(130 bud.buckets [delete] v35'6 uv6 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.097+0000 7f8eb4d73f40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:bud.buckets, ofs0, ns] 2024-09-13T23:44:36.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.097+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:131 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94104609459584 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe85c00 con 0x55966fd10000 2024-09-13T23:44:36.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.097+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 27 ==== watch-notify(notify (1) cookie 94104607798784 notify 150323855380 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x55966fd70d00 con 0x55966fd10000 2024-09-13T23:44:36.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.097+0000 7f8eadd21700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855380 cookie 94104607798784 notifier 4822 bl.length()=160 2024-09-13T23:44:36.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.097+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 28 ==== osd_op_reply(131 notify.6 [notify cookie 94104609459584 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.103 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.097+0000 7f8eadd21700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:132 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7800 con 0x55966fd10000 2024-09-13T23:44:36.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.097+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 29 ==== watch-notify(notify_complete (2) cookie 94104609459584 notify 150323855380 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55966fd70ea0 con 0x55966fd10000 2024-09-13T23:44:36.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.097+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 30 ==== osd_op_reply(132 notify.6 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.097+0000 7f8eb4d73f40 10 removing user index: bud 2024-09-13T23:44:36.104 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.097+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:133 8.4 8:267425a8:users.uid::bud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe85800 con 0x55966fd10400 2024-09-13T23:44:36.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.101+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 32 ==== osd_op_reply(133 bud [call,call,delete] v35'3 uv3 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.101+0000 7f8eb4d73f40 10 removing default.rgw.meta+users.uid+bud from cache 2024-09-13T23:44:36.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.101+0000 7f8eb4d73f40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-09-13T23:44:36.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.101+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:134 7.3 7:c609908c:::notify.5:head [notify cookie 94104609459584 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe84400 con 0x55966fd10000 2024-09-13T23:44:36.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.101+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 31 ==== watch-notify(notify (1) cookie 94104607797632 notify 150323855381 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x55966fd70ea0 con 0x55966fd10000 2024-09-13T23:44:36.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.101+0000 7f8eadd21700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855381 cookie 94104607797632 notifier 4822 bl.length()=152 2024-09-13T23:44:36.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.101+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 32 ==== osd_op_reply(134 notify.5 [notify cookie 94104609459584 out=8b] v0'0 uv222 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.106 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.101+0000 7f8eadd21700 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:135 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda6400 con 0x55966fd10000 2024-09-13T23:44:36.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.101+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 33 ==== osd_op_reply(135 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.101+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 34 ==== watch-notify(notify_complete (2) cookie 94104609459584 notify 150323855381 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55966fd71040 con 0x55966fd10000 2024-09-13T23:44:36.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.101+0000 7f8e9a4fa700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:36.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.101+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:136 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94104609456128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd11800 con 0x55966fc55000 2024-09-13T23:44:36.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.105+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 43 ==== osd_op_reply(136 data_loggenerations_metadata [watch unwatch cookie 94104609456128] v35'222 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.105+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:137 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94104607791872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd10c00 con 0x55966fc55400 2024-09-13T23:44:36.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.109+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 40 ==== osd_op_reply(137 notify.0 [watch unwatch cookie 94104607791872] v35'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.109+0000 7f8eb4d73f40 20 remove_watcher() i=0 2024-09-13T23:44:36.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.109+0000 7f8eb4d73f40 2 removed watcher, disabling cache 2024-09-13T23:44:36.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.109+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:138 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94104607793024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe84800 con 0x55966fc55000 2024-09-13T23:44:36.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.109+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.1 v2:172.21.15.167:6800/3875661733 44 ==== osd_op_reply(138 notify.1 [watch unwatch cookie 94104607793024] v35'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fc55000 2024-09-13T23:44:36.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.109+0000 7f8eb4d73f40 20 remove_watcher() i=1 2024-09-13T23:44:36.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.109+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:139 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94104607794176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe85000 con 0x55966fc55400 2024-09-13T23:44:36.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.113+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 41 ==== osd_op_reply(139 notify.2 [watch unwatch cookie 94104607794176] v35'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.113+0000 7f8eb4d73f40 20 remove_watcher() i=2 2024-09-13T23:44:36.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.113+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:140 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94104607795328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fe85c00 con 0x55966fd10400 2024-09-13T23:44:36.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.113+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 33 ==== osd_op_reply(140 notify.3 [watch unwatch cookie 94104607795328] v35'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.113+0000 7f8eb4d73f40 20 remove_watcher() i=3 2024-09-13T23:44:36.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.113+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:141 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94104607796480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fc55c00 con 0x55966fc55400 2024-09-13T23:44:36.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.117+0000 7f8eb352c700 1 -- 172.21.15.167:0/55888693 <== osd.0 v2:172.21.15.12:6800/2649399468 42 ==== osd_op_reply(141 notify.4 [watch unwatch cookie 94104607796480] v35'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1200 con 0x55966fc55400 2024-09-13T23:44:36.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.117+0000 7f8eb4d73f40 20 remove_watcher() i=4 2024-09-13T23:44:36.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.117+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:142 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94104607797632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fd13400 con 0x55966fd10000 2024-09-13T23:44:36.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.121+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 35 ==== osd_op_reply(142 notify.5 [watch unwatch cookie 94104607797632] v35'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.121+0000 7f8eb4d73f40 20 remove_watcher() i=5 2024-09-13T23:44:36.124 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.121+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:143 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94104607798784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7c00 con 0x55966fd10000 2024-09-13T23:44:36.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.121+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 <== osd.3 v2:172.21.15.167:6816/3688363524 36 ==== osd_op_reply(143 notify.6 [watch unwatch cookie 94104607798784] v35'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb1440 con 0x55966fd10000 2024-09-13T23:44:36.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.121+0000 7f8eb4d73f40 20 remove_watcher() i=6 2024-09-13T23:44:36.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.121+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:144 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94104607799936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55966fda7000 con 0x55966fd10400 2024-09-13T23:44:36.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb3d2d700 1 -- 172.21.15.167:0/55888693 <== osd.2 v2:172.21.15.167:6808/1180369139 34 ==== osd_op_reply(144 notify.7 [watch unwatch cookie 94104607799936] v35'224 uv222 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55966fcb0b40 con 0x55966fd10400 2024-09-13T23:44:36.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 20 remove_watcher() i=7 2024-09-13T23:44:36.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55966fc55400 msgr2=0x55966fb8c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55966fc55400 0x55966fb8c100 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55966fc55000 msgr2=0x55966fb8bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55966fc55000 0x55966fb8bb80 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55966fd10400 msgr2=0x55966fb8cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55966fd10400 0x55966fb8cc00 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55966fd10000 msgr2=0x55966fb8c680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55966fd10000 0x55966fb8c680 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fc54c00 msgr2=0x55966fb8b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fc54c00 0x55966fb8b600 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55966fc53b30 tx=0x55966fc09f80 comp rx=0 tx=0).stop 2024-09-13T23:44:36.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 msgr2=0x55966fb8ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 0x55966fb8ab00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x5596707182d0 tx=0x55966f9a9b60 comp rx=0 tx=0).stop 2024-09-13T23:44:36.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb2d2b700 1 -- 172.21.15.167:0/55888693 reap_dead start 2024-09-13T23:44:36.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 shutdown_connections 2024-09-13T23:44:36.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/55888693 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5800 0x55966fb8ab00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 >> 172.21.15.167:0/55888693 conn(0x55966f8a5400 msgr2=0x55966f9a5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:36.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 shutdown_connections 2024-09-13T23:44:36.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/55888693 wait complete. 2024-09-13T23:44:36.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55966fd13000 msgr2=0x55966fb8d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55966fd13000 0x55966fb8d700 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55966fc55800 msgr2=0x55966fdad080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55966fc55800 0x55966fdad080 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55966fd12000 msgr2=0x55966fdac580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55966fd12000 0x55966fdac580 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55966fd10800 msgr2=0x55966fdacb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55966fd10800 0x55966fdacb00 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fda6000 msgr2=0x55966fdac000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fda6000 0x55966fdac000 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55966fc52e70 tx=0x55966fc093e0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 msgr2=0x55966fb8d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 0x55966fb8d180 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x55966fd23350 tx=0x55966fd242a0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb352c700 1 -- 172.21.15.167:0/1927905630 reap_dead start 2024-09-13T23:44:36.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 shutdown_connections 2024-09-13T23:44:36.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/1927905630 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966fd12c00 0x55966fb8d180 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 >> 172.21.15.167:0/1927905630 conn(0x55966fd13c00 msgr2=0x55966f9a6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:36.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 shutdown_connections 2024-09-13T23:44:36.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/1927905630 wait complete. 2024-09-13T23:44:36.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/817330095 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fc54000 msgr2=0x55966fb8b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/817330095 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fc54000 0x55966fb8b080 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5596707190e0 tx=0x55966fc08120 comp rx=0 tx=0).stop 2024-09-13T23:44:36.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/817330095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 msgr2=0x55966fb8a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/817330095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x559670709f20 tx=0x55966f9a9920 comp rx=0 tx=0).stop 2024-09-13T23:44:36.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/817330095 shutdown_connections 2024-09-13T23:44:36.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/817330095 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55966fc54000 0x55966fb8b080 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 --2- 172.21.15.167:0/817330095 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55966f8a5000 0x55966fb8a580 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/817330095 >> 172.21.15.167:0/817330095 conn(0x55966f8a4c00 msgr2=0x55966f9a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:36.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/817330095 shutdown_connections 2024-09-13T23:44:36.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.125+0000 7f8eb4d73f40 1 -- 172.21.15.167:0/817330095 wait complete. 2024-09-13T23:44:36.136 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud', '--purge-data'] 2024-09-13T23:44:36.136 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud', '--purge-data'] 2024-09-13T23:44:36.137 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud --purge-data 2024-09-13T23:44:36.222 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:44:36.222 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:44:36.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.217+0000 7f11410bef40 1 Processor -- start 2024-09-13T23:44:36.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.217+0000 7f11410bef40 1 -- start start 2024-09-13T23:44:36.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.217+0000 7f11410bef40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x562e9a2f9000 0x562e9a5de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.217+0000 7f11410bef40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x562e9a3bd500 con 0x562e9a2f9400 2024-09-13T23:44:36.223 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.217+0000 7f11410bef40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x562e9a386a00 con 0x562e9a2f9000 2024-09-13T23:44:36.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.217+0000 7f113f877700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x562e9a2f9000 0x562e9a5de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.217+0000 7f113f076700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x562e9a2f9400 0x562e9a3f8800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:36476/0 (socket says 172.21.15.167:36476) 2024-09-13T23:44:36.224 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.217+0000 7f113f076700 1 -- 172.21.15.167:0/732298278 learned_addr learned my addr 172.21.15.167:0/732298278 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:44:36.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f113e875700 1 -- 172.21.15.167:0/732298278 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838982393 0 0) 0x562e9a454000 con 0x562e9a2f9400 2024-09-13T23:44:36.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f113e875700 1 -- 172.21.15.167:0/732298278 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x562e9a3bd680 con 0x562e9a2f9400 2024-09-13T23:44:36.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f113f877700 1 -- 172.21.15.167:0/732298278 >> v1:172.21.15.12:6789/0 conn(0x562e9a2f9400 legacy=0x562e9a3f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f113f877700 1 -- 172.21.15.167:0/732298278 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e9a30bd40 con 0x562e9a2f9000 2024-09-13T23:44:36.225 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f113f877700 1 --2- 172.21.15.167:0/732298278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x562e9a5d9a70 tx=0x562e9a3fd980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88ff7fd0586f0a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f113e875700 1 -- 172.21.15.167:0/732298278 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a39d4a0 con 0x562e9a2f9000 2024-09-13T23:44:36.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f113e875700 1 -- 172.21.15.167:0/732298278 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e9a3bd500 con 0x562e9a2f9000 2024-09-13T23:44:36.226 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f113e875700 1 -- 172.21.15.167:0/732298278 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a39d600 con 0x562e9a2f9000 2024-09-13T23:44:36.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f11410bef40 1 -- 172.21.15.167:0/732298278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 msgr2=0x562e9a5de580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f11410bef40 1 --2- 172.21.15.167:0/732298278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x562e9a5d9a70 tx=0x562e9a3fd980 comp rx=0 tx=0).stop 2024-09-13T23:44:36.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f11410bef40 1 -- 172.21.15.167:0/732298278 shutdown_connections 2024-09-13T23:44:36.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f11410bef40 1 --2- 172.21.15.167:0/732298278 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f11410bef40 1 -- 172.21.15.167:0/732298278 >> 172.21.15.167:0/732298278 conn(0x562e9a2f8c00 msgr2=0x562e9a3f8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:36.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f11410bef40 1 -- 172.21.15.167:0/732298278 shutdown_connections 2024-09-13T23:44:36.227 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f11410bef40 1 -- 172.21.15.167:0/732298278 wait complete. 2024-09-13T23:44:36.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f11410bef40 1 Processor -- start 2024-09-13T23:44:36.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f11410bef40 1 -- start start 2024-09-13T23:44:36.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f11410bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f11410bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e9a386500 con 0x562e9a2f9000 2024-09-13T23:44:36.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f113f877700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f113f877700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43090/0 (socket says 172.21.15.167:43090) 2024-09-13T23:44:36.228 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.221+0000 7f113f877700 1 -- 172.21.15.167:0/2304265789 learned_addr learned my addr 172.21.15.167:0/2304265789 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:36.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f113f877700 1 -- 172.21.15.167:0/2304265789 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e9a30bba0 con 0x562e9a2f9000 2024-09-13T23:44:36.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f113f877700 1 --2- 172.21.15.167:0/2304265789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x562e9b0cde00 tx=0x562e9a3fd800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e19fb3cff133729 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f113e074700 1 -- 172.21.15.167:0/2304265789 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a39d760 con 0x562e9a2f9000 2024-09-13T23:44:36.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f113e074700 1 -- 172.21.15.167:0/2304265789 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e9a3bd980 con 0x562e9a2f9000 2024-09-13T23:44:36.229 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f113e074700 1 -- 172.21.15.167:0/2304265789 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a39d1e0 con 0x562e9a2f9000 2024-09-13T23:44:36.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f11410bef40 1 -- 172.21.15.167:0/2304265789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 msgr2=0x562e9a5de580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f11410bef40 1 --2- 172.21.15.167:0/2304265789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x562e9b0cde00 tx=0x562e9a3fd800 comp rx=0 tx=0).stop 2024-09-13T23:44:36.230 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f11410bef40 1 -- 172.21.15.167:0/2304265789 shutdown_connections 2024-09-13T23:44:36.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f11410bef40 1 --2- 172.21.15.167:0/2304265789 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f11410bef40 1 -- 172.21.15.167:0/2304265789 >> 172.21.15.167:0/2304265789 conn(0x562e9a2f8c00 msgr2=0x562e9a3f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:36.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f11410bef40 1 -- 172.21.15.167:0/2304265789 shutdown_connections 2024-09-13T23:44:36.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f11410bef40 1 -- 172.21.15.167:0/2304265789 wait complete. 2024-09-13T23:44:36.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f11410bef40 1 Processor -- start 2024-09-13T23:44:36.231 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f11410bef40 1 -- start start 2024-09-13T23:44:36.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f11410bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f11410bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e9a3863c0 con 0x562e9a2f9000 2024-09-13T23:44:36.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f113f877700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f113f877700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43104/0 (socket says 172.21.15.167:43104) 2024-09-13T23:44:36.232 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.225+0000 7f113f877700 1 -- 172.21.15.167:0/1403426715 learned_addr learned my addr 172.21.15.167:0/1403426715 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:36.233 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f113f877700 1 -- 172.21.15.167:0/1403426715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e9a30ba00 con 0x562e9a2f9000 2024-09-13T23:44:36.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f113f877700 1 --2- 172.21.15.167:0/1403426715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x562e9a5d9f80 tx=0x562e9a3fd980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.234 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f113c871700 1 -- 172.21.15.167:0/1403426715 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a39d4a0 con 0x562e9a2f9000 2024-09-13T23:44:36.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f113c871700 1 -- 172.21.15.167:0/1403426715 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e9a3bdb00 con 0x562e9a2f9000 2024-09-13T23:44:36.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f11410bef40 1 -- 172.21.15.167:0/1403426715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562e9a30b860 con 0x562e9a2f9000 2024-09-13T23:44:36.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f113c871700 1 -- 172.21.15.167:0/1403426715 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a39d600 con 0x562e9a2f9000 2024-09-13T23:44:36.235 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f11410bef40 1 -- 172.21.15.167:0/1403426715 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e9a30b6c0 con 0x562e9a2f9000 2024-09-13T23:44:36.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f11410bef40 1 Processor -- start 2024-09-13T23:44:36.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f11410bef40 1 -- start start 2024-09-13T23:44:36.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f11410bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 0x562e9a5deb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f11410bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e9a31e640 con 0x562e9a2f9800 2024-09-13T23:44:36.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f1140078700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 0x562e9a5deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f1140078700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 0x562e9a5deb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43108/0 (socket says 172.21.15.167:43108) 2024-09-13T23:44:36.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.229+0000 7f1140078700 1 -- 172.21.15.167:0/863813338 learned_addr learned my addr 172.21.15.167:0/863813338 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:36.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f113c871700 1 -- 172.21.15.167:0/1403426715 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x562e9a3f2b00 con 0x562e9a2f9000 2024-09-13T23:44:36.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f113c871700 1 --2- 172.21.15.167:0/1403426715 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a6b6000 0x562e9a5df080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.237 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f113c871700 1 -- 172.21.15.167:0/1403426715 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x562e9a35f180 con 0x562e9a2f9000 2024-09-13T23:44:36.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f113f877700 1 --2- 172.21.15.167:0/1403426715 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a6b6000 0x562e9a5df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.238 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f1140078700 1 -- 172.21.15.167:0/863813338 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e9a30a000 con 0x562e9a2f9800 2024-09-13T23:44:36.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f1140078700 1 --2- 172.21.15.167:0/863813338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 0x562e9a5deb00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x562e9a6375f0 tx=0x562e9a66ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975064e260df77ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f113f877700 1 --2- 172.21.15.167:0/1403426715 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a6b6000 0x562e9a5df080 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x562e9a637950 tx=0x562e9a66e120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f113a06c700 1 -- 172.21.15.167:0/863813338 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a39d4a0 con 0x562e9a2f9800 2024-09-13T23:44:36.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f113a06c700 1 -- 172.21.15.167:0/863813338 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e9a3bdb00 con 0x562e9a2f9800 2024-09-13T23:44:36.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f113a06c700 1 -- 172.21.15.167:0/863813338 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a39d600 con 0x562e9a2f9800 2024-09-13T23:44:36.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f11410bef40 1 -- 172.21.15.167:0/863813338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 msgr2=0x562e9a5deb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f11410bef40 1 --2- 172.21.15.167:0/863813338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 0x562e9a5deb00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x562e9a6375f0 tx=0x562e9a66ed20 comp rx=0 tx=0).stop 2024-09-13T23:44:36.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f11410bef40 1 -- 172.21.15.167:0/863813338 shutdown_connections 2024-09-13T23:44:36.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f11410bef40 1 --2- 172.21.15.167:0/863813338 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 0x562e9a5deb00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.240 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f11410bef40 1 -- 172.21.15.167:0/863813338 >> 172.21.15.167:0/863813338 conn(0x562e9a2f9400 msgr2=0x562e9a3f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:36.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f11410bef40 1 -- 172.21.15.167:0/863813338 shutdown_connections 2024-09-13T23:44:36.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f11410bef40 1 -- 172.21.15.167:0/863813338 wait complete. 2024-09-13T23:44:36.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f11410bef40 1 Processor -- start 2024-09-13T23:44:36.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f11410bef40 1 -- start start 2024-09-13T23:44:36.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f11410bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 0x562e9a5deb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f11410bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e9a31e8c0 con 0x562e9a2f9800 2024-09-13T23:44:36.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f1140078700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 0x562e9a5deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f1140078700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 0x562e9a5deb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43118/0 (socket says 172.21.15.167:43118) 2024-09-13T23:44:36.241 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.233+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 learned_addr learned my addr 172.21.15.167:0/42528590 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:36.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e9a30aea0 con 0x562e9a2f9800 2024-09-13T23:44:36.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f1140078700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 0x562e9a5deb00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x562e9a6361b0 tx=0x562e9a3fdda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f1138869700 1 -- 172.21.15.167:0/42528590 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a39d760 con 0x562e9a2f9800 2024-09-13T23:44:36.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f1138869700 1 -- 172.21.15.167:0/42528590 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e9a668000 con 0x562e9a2f9800 2024-09-13T23:44:36.242 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f1138869700 1 -- 172.21.15.167:0/42528590 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a39d4a0 con 0x562e9a2f9800 2024-09-13T23:44:36.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562e9a30a820 con 0x562e9a2f9800 2024-09-13T23:44:36.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e9a30a9c0 con 0x562e9a2f9800 2024-09-13T23:44:36.243 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f1138869700 1 -- 172.21.15.167:0/42528590 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x562e9a3f3080 con 0x562e9a2f9800 2024-09-13T23:44:36.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f1138869700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a6b6c00 0x562e9a5df600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f1138869700 1 -- 172.21.15.167:0/42528590 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x562e9a35f340 con 0x562e9a2f9800 2024-09-13T23:44:36.244 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f113f076700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a6b6c00 0x562e9a5df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f11410bef40 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562e9a6b7000 0x562e9a5dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.237+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7400 con 0x562e9a6b7000 2024-09-13T23:44:36.245 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f1140078700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562e9a6b7000 0x562e9a5dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f113f076700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a6b6c00 0x562e9a5df600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x562e9b0cc030 tx=0x562e9a66ff80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.246 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f1140078700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562e9a6b7000 0x562e9a5dfb80 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.247 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f11410bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:36.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f11410bef40 20 realm 2024-09-13T23:44:36.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7000 2024-09-13T23:44:36.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f11410bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:36.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f11410bef40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:44:36.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7c00 con 0x562e9a6b7000 2024-09-13T23:44:36.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.248 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f11410bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:36.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f11410bef40 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562e9a6b7400 0x562e9a5e0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7400 2024-09-13T23:44:36.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f113f877700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562e9a6b7400 0x562e9a5e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.249 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.241+0000 7f113f877700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562e9a6b7400 0x562e9a5e0100 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f11410bef40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:36.250 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7c00 con 0x562e9a6b7400 2024-09-13T23:44:36.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f11410bef40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:44:36.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f11410bef40 20 searching for the correct realm 2024-09-13T23:44:36.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f11410bef40 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562e9a764000 0x562e9a5e0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a764400 con 0x562e9a764000 2024-09-13T23:44:36.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f113f076700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562e9a764000 0x562e9a5e0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.251 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f113f076700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562e9a764000 0x562e9a5e0680 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a764400 con 0x562e9a6b7400 2024-09-13T23:44:36.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f113f877700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562e9a764400 0x562e9a5e0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.245+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764400 2024-09-13T23:44:36.252 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.249+0000 7f1140078700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562e9a764400 0x562e9a5e0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.253 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.249+0000 7f1140078700 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562e9a764400 0x562e9a5e0c00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.249+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.249+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7000 2024-09-13T23:44:36.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.249+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.254 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.249+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7000 2024-09-13T23:44:36.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.249+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.249+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764000 2024-09-13T23:44:36.255 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.249+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.249+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7000 2024-09-13T23:44:36.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.249+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.249+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7400 2024-09-13T23:44:36.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7400 2024-09-13T23:44:36.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.257 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7000 2024-09-13T23:44:36.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764400 2024-09-13T23:44:36.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7000 2024-09-13T23:44:36.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764400 2024-09-13T23:44:36.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.259 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7400 2024-09-13T23:44:36.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.253+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764000 2024-09-13T23:44:36.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.257+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.257+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7000 2024-09-13T23:44:36.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.257+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.257+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7400 2024-09-13T23:44:36.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.257+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.257+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764000 2024-09-13T23:44:36.262 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.257+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.257+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7000 2024-09-13T23:44:36.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.257+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.257+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764400 2024-09-13T23:44:36.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.257+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.257+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7000 2024-09-13T23:44:36.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764000 2024-09-13T23:44:36.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.265 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764400 2024-09-13T23:44:36.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7000 2024-09-13T23:44:36.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764400 2024-09-13T23:44:36.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764000 2024-09-13T23:44:36.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764400 2024-09-13T23:44:36.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.261+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7400 2024-09-13T23:44:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.265+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.265+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7400 2024-09-13T23:44:36.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.265+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.265+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764000 2024-09-13T23:44:36.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.265+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.265+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7400 2024-09-13T23:44:36.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.265+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.265+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a6b7400 2024-09-13T23:44:36.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:44:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:44:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:44:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:44:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a764800 con 0x562e9a6b7000 2024-09-13T23:44:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a764c00 con 0x562e9a6b7400 2024-09-13T23:44:36.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:44:36.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765000 con 0x562e9a6b7400 2024-09-13T23:44:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:44:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 zone default found 2024-09-13T23:44:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 4 Realm: () 2024-09-13T23:44:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:44:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:44:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:44:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 zonegroup default 2024-09-13T23:44:36.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765400 con 0x562e9a764400 2024-09-13T23:44:36.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:36.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.269+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765800 con 0x562e9a764000 2024-09-13T23:44:36.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.273+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.273+0000 7f11410bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:36.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.273+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.273+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765c00 con 0x562e9a764000 2024-09-13T23:44:36.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.273+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.273+0000 7f11410bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:36.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.273+0000 7f11410bef40 20 started sync module instance, tier type = 2024-09-13T23:44:36.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.273+0000 7f11410bef40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:44:36.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.273+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765800 con 0x562e9a6b7400 2024-09-13T23:44:36.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.277+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.277+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94758158293760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765c00 con 0x562e9a6b7400 2024-09-13T23:44:36.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.277+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a764c00 con 0x562e9a6b7000 2024-09-13T23:44:36.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.277+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94758158293760] v35'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.277+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.277+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94758158294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765000 con 0x562e9a6b7000 2024-09-13T23:44:36.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.277+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765800 con 0x562e9a6b7400 2024-09-13T23:44:36.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.281+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94758158294912] v35'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.281+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.281+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94758158296064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7c00 con 0x562e9a6b7400 2024-09-13T23:44:36.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.281+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7800 con 0x562e9a764400 2024-09-13T23:44:36.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.285+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94758158296064] v35'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.285+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.285+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94758158297216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a764800 con 0x562e9a764400 2024-09-13T23:44:36.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.285+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765400 con 0x562e9a6b7400 2024-09-13T23:44:36.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.289+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94758158297216] v35'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.289+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.289+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94758158298368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a764c00 con 0x562e9a6b7400 2024-09-13T23:44:36.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.289+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765000 con 0x562e9a764000 2024-09-13T23:44:36.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.289+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.289+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94758158299520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76e000 con 0x562e9a764000 2024-09-13T23:44:36.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.289+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76e400 con 0x562e9a764000 2024-09-13T23:44:36.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.289+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94758158298368] v35'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94758158299520] v35'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94758158300672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76e800 con 0x562e9a764000 2024-09-13T23:44:36.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76ec00 con 0x562e9a764400 2024-09-13T23:44:36.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94758158300672] v35'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'225 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94758158301824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76f000 con 0x562e9a764400 2024-09-13T23:44:36.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f11410bef40 20 add_watcher() i=0 2024-09-13T23:44:36.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f11410bef40 20 add_watcher() i=1 2024-09-13T23:44:36.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f11410bef40 20 add_watcher() i=2 2024-09-13T23:44:36.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f11410bef40 20 add_watcher() i=3 2024-09-13T23:44:36.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f11410bef40 20 add_watcher() i=4 2024-09-13T23:44:36.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f11410bef40 20 add_watcher() i=5 2024-09-13T23:44:36.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.293+0000 7f11410bef40 20 add_watcher() i=6 2024-09-13T23:44:36.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.297+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94758158301824] v35'226 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.297+0000 7f11410bef40 20 add_watcher() i=7 2024-09-13T23:44:36.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.297+0000 7f11410bef40 2 all 8 watchers are set, enabling cache 2024-09-13T23:44:36.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.297+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76f400 con 0x562e9a6b7000 2024-09-13T23:44:36.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.297+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.297+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94758159990784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76f800 con 0x562e9a6b7000 2024-09-13T23:44:36.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.297+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94758159990784] v35'223 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.297+0000 7f1126845700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:36.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f11410bef40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:44:36.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f11410bef40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:44:36.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f11410bef40 1 Processor -- start 2024-09-13T23:44:36.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f11410bef40 1 -- start start 2024-09-13T23:44:36.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f11410bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 0x562e9a5e1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f11410bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e9a3863c0 con 0x562e9a76ec00 2024-09-13T23:44:36.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f113f076700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 0x562e9a5e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f113f076700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 0x562e9a5e1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43120/0 (socket says 172.21.15.167:43120) 2024-09-13T23:44:36.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f113f076700 1 -- 172.21.15.167:0/941112857 learned_addr learned my addr 172.21.15.167:0/941112857 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:36.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f113f076700 1 -- 172.21.15.167:0/941112857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e9a30ab60 con 0x562e9a76ec00 2024-09-13T23:44:36.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f113f076700 1 --2- 172.21.15.167:0/941112857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 0x562e9a5e1180 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x562e9a77c0c0 tx=0x562e9a77e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44687e52beda081d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f1126044700 1 -- 172.21.15.167:0/941112857 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a782000 con 0x562e9a76ec00 2024-09-13T23:44:36.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f1126044700 1 -- 172.21.15.167:0/941112857 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e9a668480 con 0x562e9a76ec00 2024-09-13T23:44:36.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.301+0000 7f1126044700 1 -- 172.21.15.167:0/941112857 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a782160 con 0x562e9a76ec00 2024-09-13T23:44:36.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 -- 172.21.15.167:0/941112857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 msgr2=0x562e9a5e1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 --2- 172.21.15.167:0/941112857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 0x562e9a5e1180 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x562e9a77c0c0 tx=0x562e9a77e000 comp rx=0 tx=0).stop 2024-09-13T23:44:36.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 -- 172.21.15.167:0/941112857 shutdown_connections 2024-09-13T23:44:36.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 --2- 172.21.15.167:0/941112857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 0x562e9a5e1180 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 -- 172.21.15.167:0/941112857 >> 172.21.15.167:0/941112857 conn(0x562e9a76fc00 msgr2=0x562e9a3fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:36.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 -- 172.21.15.167:0/941112857 shutdown_connections 2024-09-13T23:44:36.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 -- 172.21.15.167:0/941112857 wait complete. 2024-09-13T23:44:36.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 Processor -- start 2024-09-13T23:44:36.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 -- start start 2024-09-13T23:44:36.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 0x562e9a5e1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e9a386500 con 0x562e9a76ec00 2024-09-13T23:44:36.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f113f076700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 0x562e9a5e1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f113f076700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 0x562e9a5e1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43128/0 (socket says 172.21.15.167:43128) 2024-09-13T23:44:36.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f113f076700 1 -- 172.21.15.167:0/51888767 learned_addr learned my addr 172.21.15.167:0/51888767 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:44:36.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f113f076700 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e9a30ad00 con 0x562e9a76ec00 2024-09-13T23:44:36.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f113f076700 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 0x562e9a5e1180 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x562e9a77d380 tx=0x562e9a77e2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.311 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f1124841700 1 -- 172.21.15.167:0/51888767 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a7822c0 con 0x562e9a76ec00 2024-09-13T23:44:36.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f1124841700 1 -- 172.21.15.167:0/51888767 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e9a668600 con 0x562e9a76ec00 2024-09-13T23:44:36.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562e9a30a340 con 0x562e9a76ec00 2024-09-13T23:44:36.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f1124841700 1 -- 172.21.15.167:0/51888767 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x562e9a782000 con 0x562e9a76ec00 2024-09-13T23:44:36.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.305+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e9a30ab60 con 0x562e9a76ec00 2024-09-13T23:44:36.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.309+0000 7f1124841700 1 -- 172.21.15.167:0/51888767 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x562e9a3f3b80 con 0x562e9a76ec00 2024-09-13T23:44:36.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.309+0000 7f1124841700 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a7fe000 0x562e9a804000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.309+0000 7f1124841700 1 -- 172.21.15.167:0/51888767 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x562e9a35f500 con 0x562e9a76ec00 2024-09-13T23:44:36.313 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.309+0000 7f1140078700 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a7fe000 0x562e9a804000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.309+0000 7f11410bef40 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562e9a76f000 0x562e9a5e1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.309+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76f400 con 0x562e9a76f000 2024-09-13T23:44:36.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.309+0000 7f1140078700 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a7fe000 0x562e9a804000 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x562e9a707800 tx=0x562e9a66f3e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.309+0000 7f113f877700 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562e9a76f000 0x562e9a5e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.314 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.309+0000 7f113f877700 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562e9a76f000 0x562e9a5e1700 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.313+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v35'39 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.313+0000 7f11410bef40 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562e9a76e000 0x562e9a804580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.313+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76e400 con 0x562e9a76e000 2024-09-13T23:44:36.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.313+0000 7f113f076700 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562e9a76e000 0x562e9a804580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.313+0000 7f113f076700 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562e9a76e000 0x562e9a804580 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.317+0000 7f113f076700 1 -- 172.21.15.167:0/51888767 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v35'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a76e000 2024-09-13T23:44:36.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.317+0000 7f11410bef40 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562e9a764800 0x562e9a804b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.317+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765000 con 0x562e9a764800 2024-09-13T23:44:36.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.317+0000 7f1140078700 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562e9a764800 0x562e9a804b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.317+0000 7f1140078700 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562e9a764800 0x562e9a804b00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.317+0000 7f1140078700 1 -- 172.21.15.167:0/51888767 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v35'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764800 2024-09-13T23:44:36.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.317+0000 7f11410bef40 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562e9a6b7800 0x562e9a805080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:44:36.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.317+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe400 con 0x562e9a6b7800 2024-09-13T23:44:36.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.317+0000 7f113f877700 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562e9a6b7800 0x562e9a805080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:44:36.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.321+0000 7f113f877700 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562e9a6b7800 0x562e9a805080 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:44:36.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.321+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v35'224 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7800 2024-09-13T23:44:36.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.321+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe800 con 0x562e9a76f000 2024-09-13T23:44:36.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.325+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v35'71 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.325+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fec00 con 0x562e9a764800 2024-09-13T23:44:36.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.329+0000 7f1140078700 1 -- 172.21.15.167:0/51888767 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v35'50 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764800 2024-09-13T23:44:36.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.329+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff000 con 0x562e9a76f000 2024-09-13T23:44:36.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.329+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v35'29 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.329+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff400 con 0x562e9a6b7800 2024-09-13T23:44:36.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.333+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v35'48 uv35 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7800 2024-09-13T23:44:36.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.333+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff800 con 0x562e9a76f000 2024-09-13T23:44:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.337+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v35'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.337+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ffc00 con 0x562e9a76f000 2024-09-13T23:44:36.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.337+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v35'34 uv28 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.337+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76f400 con 0x562e9a76f000 2024-09-13T23:44:36.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.341+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v35'72 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.347 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.341+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765c00 con 0x562e9a764800 2024-09-13T23:44:36.349 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.345+0000 7f1140078700 1 -- 172.21.15.167:0/51888767 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v35'31 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764800 2024-09-13T23:44:36.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.345+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765800 con 0x562e9a76f000 2024-09-13T23:44:36.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.349+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v35'30 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.349+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765400 con 0x562e9a6b7800 2024-09-13T23:44:36.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.349+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v35'54 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7800 2024-09-13T23:44:36.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.349+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a764c00 con 0x562e9a6b7800 2024-09-13T23:44:36.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.353+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v35'225 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7800 2024-09-13T23:44:36.358 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.353+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7c00 con 0x562e9a76f000 2024-09-13T23:44:36.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.357+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v35'68 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.361 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.357+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76f400 con 0x562e9a76f000 2024-09-13T23:44:36.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.357+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v35'73 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.364 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.361+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe400 con 0x562e9a764800 2024-09-13T23:44:36.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.361+0000 7f1140078700 1 -- 172.21.15.167:0/51888767 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v35'29 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764800 2024-09-13T23:44:36.367 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.361+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe800 con 0x562e9a76f000 2024-09-13T23:44:36.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.365+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v35'36 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.365+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff000 con 0x562e9a6b7800 2024-09-13T23:44:36.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.369+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v35'226 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7800 2024-09-13T23:44:36.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.369+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff400 con 0x562e9a76f000 2024-09-13T23:44:36.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.373+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v35'69 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.376 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.373+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff800 con 0x562e9a76f000 2024-09-13T23:44:36.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.373+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v35'37 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.379 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.373+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ffc00 con 0x562e9a764800 2024-09-13T23:44:36.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.377+0000 7f1140078700 1 -- 172.21.15.167:0/51888767 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v35'29 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764800 2024-09-13T23:44:36.382 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.377+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fec00 con 0x562e9a76f000 2024-09-13T23:44:36.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.381+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v35'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.385 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.381+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe400 con 0x562e9a764800 2024-09-13T23:44:36.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.381+0000 7f1140078700 1 -- 172.21.15.167:0/51888767 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v35'23 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764800 2024-09-13T23:44:36.388 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.381+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ffc00 con 0x562e9a764800 2024-09-13T23:44:36.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.385+0000 7f1140078700 1 -- 172.21.15.167:0/51888767 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v35'51 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764800 2024-09-13T23:44:36.390 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.385+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe800 con 0x562e9a764800 2024-09-13T23:44:36.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.389+0000 7f1140078700 1 -- 172.21.15.167:0/51888767 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v35'29 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764800 2024-09-13T23:44:36.393 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.389+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff000 con 0x562e9a76f000 2024-09-13T23:44:36.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.389+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v35'27 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.396 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.389+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff400 con 0x562e9a6b7800 2024-09-13T23:44:36.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.393+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v35'49 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7800 2024-09-13T23:44:36.399 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.393+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76f800 con 0x562e9a76f000 2024-09-13T23:44:36.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.397+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v35'70 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a76f000 2024-09-13T23:44:36.402 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.397+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76f400 con 0x562e9a6b7800 2024-09-13T23:44:36.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.397+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v35'55 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7800 2024-09-13T23:44:36.404 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.401+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7c00 con 0x562e9a76e000 2024-09-13T23:44:36.406 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.401+0000 7f113f076700 1 -- 172.21.15.167:0/51888767 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v35'33 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a76e000 2024-09-13T23:44:36.409 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f112203c700 20 reqs_thread_entry: start 2024-09-13T23:44:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 20 init_complete bucket index max shards: 11 2024-09-13T23:44:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:44:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f1120038700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:45:06 2024 2024-09-13T23:44:36.410 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:44:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f111f816700 20 reqs_thread_entry: start 2024-09-13T23:44:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-09-13T23:44:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.410 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765800 con 0x562e9a764400 2024-09-13T23:44:36.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(62 fud [call out=48b,read 0~346 out=346b,getxattrs out=75b] v0'0 uv5 ondisk = 0) v8 ==== 231+0+469 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 20 rados_obj.operate() r=0 bl.length=346 2024-09-13T23:44:36.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-09-13T23:44:36.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-09-13T23:44:36.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:44:36.411 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765400 con 0x562e9a6b7000 2024-09-13T23:44:36.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(63 fud.buckets [call out=184b] v0'0 uv5 ondisk = 0) v8 ==== 155+0+184 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 : miss 2024-09-13T23:44:36.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.412 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.405+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 8.14 8:2e51ebcc:root::.bucket.meta.myfoo7%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a764c00 con 0x562e9a6b7400 2024-09-13T23:44:36.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.409+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 .bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 [call out=48b,stat out=16b,read 0~353 out=353b,getxattrs out=218b] v0'0 uv5 ondisk = 0) v8 ==== 333+0+635 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.409+0000 7f11410bef40 20 rados_obj.operate() r=0 bl.length=353 2024-09-13T23:44:36.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.409+0000 7f11410bef40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 info.flags=0x17 2024-09-13T23:44:36.413 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.409+0000 7f11410bef40 10 adding default.rgw.meta+root+.bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 to cache LRU end 2024-09-13T23:44:36.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.409+0000 7f11410bef40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:44:36.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.409+0000 7f11410bef40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:44:36.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.409+0000 7f11410bef40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 2024-09-13T23:44:36.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.409+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:65 4.14 4:2a26ccbf:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934000 con 0x562e9a764000 2024-09-13T23:44:36.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.409+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(65 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.0 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.414 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.409+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 4.3c 4:3d25fd75:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934400 con 0x562e9a6b7000 2024-09-13T23:44:36.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.409+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(66 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.1 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.415 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.409+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 4.3e 4:7ef1b486:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934800 con 0x562e9a6b7400 2024-09-13T23:44:36.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.413+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.2 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.416 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.413+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:68 4.2f 4:f6118c19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934c00 con 0x562e9a6b7000 2024-09-13T23:44:36.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.413+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(68 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.3 [call out=85b] v0'0 uv19 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.417 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.413+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:69 4.3 4:c2ef4078:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935000 con 0x562e9a6b7000 2024-09-13T23:44:36.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.413+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(69 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.413+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 4.1f 4:fa8713ef:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935400 con 0x562e9a6b7400 2024-09-13T23:44:36.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.413+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(70 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5 [call out=378b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+378 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.413+0000 7f11410bef40 10 cls_bucket_list_unordered: got fourteen 2024-09-13T23:44:36.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.413+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 4.11 4:8b9e65b1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935800 con 0x562e9a764000 2024-09-13T23:44:36.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.413+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(71 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.6 [call out=85b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.413+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 4.3 4:c0f1aca3:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935c00 con 0x562e9a6b7000 2024-09-13T23:44:36.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(72 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:73 4.37 4:eead2d2e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934000 con 0x562e9a764400 2024-09-13T23:44:36.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(73 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.8 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:74 4.2f 4:f556a1ab:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935800 con 0x562e9a6b7000 2024-09-13T23:44:36.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(74 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.9 [call out=85b] v0'0 uv20 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:75 4.3b 4:dfb5712d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934800 con 0x562e9a764000 2024-09-13T23:44:36.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(75 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f11410bef40 20 get_obj_state: rctx=0x562e9a389730 obj=myfoo7:fourteen state=0x562e9a8541e8 s->prefetch_data=0 2024-09-13T23:44:36.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:76 3.21s0 3:8412e4ca:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8_fourteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a764c00 con 0x562e9a764400 2024-09-13T23:44:36.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(76 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8_fourteen [getxattrs out=856b,stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 238+0+872 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f11410bef40 10 manifest: total_size = 8 2024-09-13T23:44:36.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f11410bef40 20 get_obj_state: setting s->obj_tag to 9144208c-3c85-4fb6-9007-2693d5a930f0.4222.11651906277108245705 2024-09-13T23:44:36.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f11410bef40 20 get_obj_state: rctx=0x562e9a389730 obj=myfoo7:fourteen state=0x562e9a8541e8 s->prefetch_data=0 2024-09-13T23:44:36.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f11410bef40 20 prepare_atomic_modification: state is not atomic. state=0x562e9a8541e8 2024-09-13T23:44:36.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f11410bef40 20 bucket index object: default.rgw.buckets.index:.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5 2024-09-13T23:44:36.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.417+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:77 4.1f 4:fa8713ef:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765800 con 0x562e9a6b7400 2024-09-13T23:44:36.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.421+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(77 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5 [stat,call,call] v35'4 uv4 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.421+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 3.21s0 3:8412e4ca:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8_fourteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765400 con 0x562e9a764400 2024-09-13T23:44:36.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(78 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8_fourteen [call] v35'2 uv2 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:79 4.1f 4:fa8713ef:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=264b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fec00 con 0x562e9a6b7400 2024-09-13T23:44:36.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f11410bef40 20 list_objects_ordered: starting attempt 1 2024-09-13T23:44:36.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f11410bef40 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-09-13T23:44:36.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:80 4.14 4:2a26ccbf:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff000 con 0x562e9a764000 2024-09-13T23:44:36.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:81 4.3c 4:3d25fd75:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff400 con 0x562e9a6b7000 2024-09-13T23:44:36.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:82 4.3e 4:7ef1b486:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe400 con 0x562e9a6b7400 2024-09-13T23:44:36.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:83 4.2f 4:f6118c19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ffc00 con 0x562e9a6b7000 2024-09-13T23:44:36.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(81 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.1 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.432 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:84 4.3 4:c2ef4078:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe800 con 0x562e9a6b7000 2024-09-13T23:44:36.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:85 4.1f 4:fa8713ef:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76e400 con 0x562e9a6b7400 2024-09-13T23:44:36.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:86 4.11 4:8b9e65b1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76f800 con 0x562e9a764000 2024-09-13T23:44:36.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(80 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.0 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.425+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:87 4.3 4:c0f1aca3:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76f400 con 0x562e9a6b7000 2024-09-13T23:44:36.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:88 4.37 4:eead2d2e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7c00 con 0x562e9a764400 2024-09-13T23:44:36.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:89 4.2f 4:f556a1ab:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935000 con 0x562e9a6b7000 2024-09-13T23:44:36.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:90 4.3b 4:dfb5712d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935c00 con 0x562e9a764000 2024-09-13T23:44:36.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(82 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.2 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(86 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.6 [call out=85b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(83 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.3 [call out=85b] v0'0 uv19 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(84 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(90 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(88 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.8 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(87 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(79 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5 [stat,call,call] v35'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 28 ==== osd_op_reply(89 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.9 [call out=85b] v0'0 uv20 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f113986b700 20 handle_completion(): completion ok for obj=fourteen 2024-09-13T23:44:36.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(85 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f11410bef40 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-09-13T23:44:36.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f11410bef40 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-09-13T23:44:36.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f11410bef40 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-09-13T23:44:36.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f11410bef40 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-09-13T23:44:36.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f11410bef40 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo7[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8])"; uploads.size()=0; is_truncated=0 2024-09-13T23:44:36.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f11410bef40 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo7[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8]) 2024-09-13T23:44:36.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f11410bef40 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo7 : miss 2024-09-13T23:44:36.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f11410bef40 20 lifecycle: rados->read ofs=0 len=0 2024-09-13T23:44:36.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.429+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:91 8.9 8:938c738d:root::myfoo7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934000 con 0x562e9a764400 2024-09-13T23:44:36.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.433+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 18 ==== osd_op_reply(91 myfoo7 [call out=48b,stat out=16b,read 0~166 out=166b,getxattrs out=75b] v0'0 uv42 ondisk = 0) v8 ==== 276+0+305 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.433+0000 7f11410bef40 20 lifecycle: rados_obj.operate() r=0 bl.length=166 2024-09-13T23:44:36.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.433+0000 7f11410bef40 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo7 info.flags=0x17 2024-09-13T23:44:36.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.433+0000 7f11410bef40 10 lifecycle: adding default.rgw.meta+root+myfoo7 to cache LRU end 2024-09-13T23:44:36.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.433+0000 7f11410bef40 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:44:36.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.433+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:92 8.14 8:2e51ebcc:root::.bucket.meta.myfoo7%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~353 in=353b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935400 con 0x562e9a6b7400 2024-09-13T23:44:36.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.433+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(92 .bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 [delete,create,call,call,writefull 0~353,setxattr (123)] v35'6 uv6 ondisk = 0) v8 ==== 417+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.433+0000 7f11410bef40 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 info.flags=0x17 2024-09-13T23:44:36.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f11410bef40 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 to cache LRU end 2024-09-13T23:44:36.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f11410bef40 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:44:36.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f11410bef40 10 lifecycle: distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8, ofs0, ns] 2024-09-13T23:44:36.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:93 7.e 7:7759931f:::notify.3:head [notify cookie 94758159994240 in=739b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934c00 con 0x562e9a764400 2024-09-13T23:44:36.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 19 ==== watch-notify(notify (1) cookie 94758158297216 notify 150323855393 ret 0) v3 ==== 769+0+0 (crc 0 0 0) 0x562e9a70b380 con 0x562e9a764400 2024-09-13T23:44:36.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 20 ==== osd_op_reply(93 notify.3 [notify cookie 94758159994240 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f113986b700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855393 cookie 94758158297216 notifier 4830 bl.length()=727 2024-09-13T23:44:36.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f113986b700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 info.flags=0x17 2024-09-13T23:44:36.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f113986b700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 to cache LRU end 2024-09-13T23:44:36.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f113986b700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:44:36.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f113986b700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:94 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff800 con 0x562e9a764400 2024-09-13T23:44:36.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 21 ==== watch-notify(notify_complete (2) cookie 94758159994240 notify 150323855393 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562e9a70b520 con 0x562e9a764400 2024-09-13T23:44:36.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 22 ==== osd_op_reply(94 notify.3 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f11410bef40 20 lifecycle: handle_bi_update(): bucket=:myfoo7[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8]): orig_sources= new_sources= 2024-09-13T23:44:36.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f11410bef40 20 lifecycle: handle_bi_update(): bucket=:myfoo7[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8]): potential sources added=[] removed=[] 2024-09-13T23:44:36.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f11410bef40 20 lifecycle: handle_bi_update(): bucket=:myfoo7[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8]): orig_dests= new_dests= 2024-09-13T23:44:36.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f11410bef40 20 lifecycle: handle_bi_update(): bucket=:myfoo7[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8]): potential dests added=[] removed=[] 2024-09-13T23:44:36.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:33 6.17 6:ee62d5d2:lc::lc.1:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff400 con 0x562e9a6b7800 2024-09-13T23:44:36.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(33 lc.1 [call] v35'8 uv8 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7800 2024-09-13T23:44:36.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.437+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:34 6.17 6:ee62d5d2:lc::lc.1:head [call rgw.lc_rm_entry in=93b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ffc00 con 0x562e9a6b7800 2024-09-13T23:44:36.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(34 lc.1 [call] v35'9 uv9 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7800 2024-09-13T23:44:36.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:35 6.17 6:ee62d5d2:lc::lc.1:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a764c00 con 0x562e9a6b7800 2024-09-13T23:44:36.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(35 lc.1 [call] v35'10 uv10 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7800 2024-09-13T23:44:36.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:95 4.14 4:2a26ccbf:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765400 con 0x562e9a764000 2024-09-13T23:44:36.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:96 4.3c 4:3d25fd75:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765800 con 0x562e9a6b7000 2024-09-13T23:44:36.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:97 4.3e 4:7ef1b486:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a764c00 con 0x562e9a6b7400 2024-09-13T23:44:36.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:98 4.2f 4:f6118c19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fec00 con 0x562e9a6b7000 2024-09-13T23:44:36.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:99 4.3 4:c2ef4078:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe400 con 0x562e9a6b7000 2024-09-13T23:44:36.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:100 4.1f 4:fa8713ef:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff400 con 0x562e9a6b7400 2024-09-13T23:44:36.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:101 4.11 4:8b9e65b1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ffc00 con 0x562e9a764000 2024-09-13T23:44:36.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:102 4.3 4:c0f1aca3:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934400 con 0x562e9a6b7000 2024-09-13T23:44:36.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:103 4.37 4:eead2d2e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935800 con 0x562e9a764400 2024-09-13T23:44:36.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:104 4.2f 4:f556a1ab:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935400 con 0x562e9a6b7000 2024-09-13T23:44:36.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:105 4.3b 4:dfb5712d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76e400 con 0x562e9a764000 2024-09-13T23:44:36.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(95 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.0 [call out=85b] v0'0 uv9 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 29 ==== osd_op_reply(96 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.1 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 30 ==== osd_op_reply(99 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.4 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(105 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== osd_op_reply(101 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.6 [call out=85b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 31 ==== osd_op_reply(102 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.441+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 32 ==== osd_op_reply(98 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.3 [call out=85b] v0'0 uv19 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.445+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(100 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5 [call out=124b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.445+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(97 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.2 [call out=85b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.445+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 23 ==== osd_op_reply(103 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.8 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.450 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.445+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 33 ==== osd_op_reply(104 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.9 [call out=85b] v0'0 uv20 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.445+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:106 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765800 con 0x562e9a6b7000 2024-09-13T23:44:36.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.445+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 34 ==== osd_op_reply(106 fud.buckets [call] v35'6 uv6 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.445+0000 7f11410bef40 10 cache get: name=default.rgw.meta+root+myfoo7 : hit (requested=0x11, cached=0x17) 2024-09-13T23:44:36.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.445+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:107 8.9 8:938c738d:root::myfoo7:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765400 con 0x562e9a764400 2024-09-13T23:44:36.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.449+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 24 ==== osd_op_reply(107 myfoo7 [call,call,delete] v35'43 uv43 ondisk = 0) v8 ==== 234+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.449+0000 7f11410bef40 10 removing default.rgw.meta+root+myfoo7 from cache 2024-09-13T23:44:36.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.449+0000 7f11410bef40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo7, ofs0, ns] 2024-09-13T23:44:36.453 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.449+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:108 7.e 7:7759931f:::notify.3:head [notify cookie 94758159994240 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935000 con 0x562e9a764400 2024-09-13T23:44:36.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.449+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 25 ==== watch-notify(notify (1) cookie 94758158297216 notify 150323855394 ret 0) v3 ==== 192+0+0 (crc 0 0 0) 0x562e9a70b520 con 0x562e9a764400 2024-09-13T23:44:36.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.449+0000 7f113a06c700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855394 cookie 94758158297216 notifier 4830 bl.length()=150 2024-09-13T23:44:36.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.449+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 26 ==== osd_op_reply(108 notify.3 [notify cookie 94758159994240 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.449+0000 7f113a06c700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a8e2000 con 0x562e9a764400 2024-09-13T23:44:36.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.449+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 27 ==== osd_op_reply(109 notify.3 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.449+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 28 ==== watch-notify(notify_complete (2) cookie 94758159994240 notify 150323855394 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562e9a70b6c0 con 0x562e9a764400 2024-09-13T23:44:36.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.449+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:110 8.14 8:2e51ebcc:root::.bucket.meta.myfoo7%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934000 con 0x562e9a6b7400 2024-09-13T23:44:36.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(110 .bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 [call,call,delete] v35'7 uv7 ondisk = 0) v8 ==== 291+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.457 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f11410bef40 10 removing default.rgw.meta+root+.bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 from cache 2024-09-13T23:44:36.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f11410bef40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo7:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8, ofs0, ns] 2024-09-13T23:44:36.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:111 7.e 7:7759931f:::notify.3:head [notify cookie 94758159994240 in=219b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934c00 con 0x562e9a764400 2024-09-13T23:44:36.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 29 ==== watch-notify(notify (1) cookie 94758158297216 notify 150323855395 ret 0) v3 ==== 249+0+0 (crc 0 0 0) 0x562e9a70b6c0 con 0x562e9a764400 2024-09-13T23:44:36.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f113a06c700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855395 cookie 94758158297216 notifier 4830 bl.length()=207 2024-09-13T23:44:36.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 30 ==== osd_op_reply(111 notify.3 [notify cookie 94758159994240 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.458 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f113a06c700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:112 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a8e2400 con 0x562e9a764400 2024-09-13T23:44:36.459 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 31 ==== watch-notify(notify_complete (2) cookie 94758159994240 notify 150323855395 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562e9a70b860 con 0x562e9a764400 2024-09-13T23:44:36.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 32 ==== osd_op_reply(112 notify.3 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:113 4.14 4:2a26ccbf:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934800 con 0x562e9a764000 2024-09-13T23:44:36.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:114 4.3c 4:3d25fd75:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935c00 con 0x562e9a6b7000 2024-09-13T23:44:36.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:115 4.3e 4:7ef1b486:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76f400 con 0x562e9a6b7400 2024-09-13T23:44:36.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:116 4.2f 4:f6118c19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76f800 con 0x562e9a6b7000 2024-09-13T23:44:36.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:117 4.3 4:c2ef4078:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe800 con 0x562e9a6b7000 2024-09-13T23:44:36.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:118 4.1f 4:fa8713ef:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff800 con 0x562e9a6b7400 2024-09-13T23:44:36.460 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:119 4.11 4:8b9e65b1:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fec00 con 0x562e9a764000 2024-09-13T23:44:36.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.453+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:120 4.3 4:c0f1aca3:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff000 con 0x562e9a6b7000 2024-09-13T23:44:36.461 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:121 4.37 4:eead2d2e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a6b7c00 con 0x562e9a764400 2024-09-13T23:44:36.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:122 4.2f 4:f556a1ab:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934400 con 0x562e9a6b7000 2024-09-13T23:44:36.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:123 4.3b 4:dfb5712d:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935400 con 0x562e9a764000 2024-09-13T23:44:36.462 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 35 ==== osd_op_reply(114 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.1 [delete] v35'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 36 ==== osd_op_reply(117 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.4 [delete] v35'3 uv3 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(119 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.6 [delete] v35'7 uv7 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 31 ==== osd_op_reply(118 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.5 [delete] v35'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 32 ==== osd_op_reply(115 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.2 [delete] v35'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.463 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== osd_op_reply(113 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.0 [delete] v35'10 uv10 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 25 ==== osd_op_reply(123 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.10 [delete] v35'2 uv2 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 37 ==== osd_op_reply(116 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.3 [delete] v35'21 uv21 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 38 ==== osd_op_reply(120 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.7 [delete] v35'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.464 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 33 ==== osd_op_reply(121 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.8 [delete] v35'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 39 ==== osd_op_reply(122 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8.9 [delete] v35'22 uv22 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f11410bef40 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo7/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 : miss 2024-09-13T23:44:36.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f11410bef40 20 rados->read ofs=0 len=0 2024-09-13T23:44:36.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.457+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:124 6.10 6:0f4d8598:::pubsub..bucket.myfoo7%2f9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935800 con 0x562e9a6b7000 2024-09-13T23:44:36.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.461+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 40 ==== osd_op_reply(124 pubsub..bucket.myfoo7/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.461+0000 7f11410bef40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:44:36.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.461+0000 7f11410bef40 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo7/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 info.flags=0x0 2024-09-13T23:44:36.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.461+0000 7f11410bef40 10 adding default.rgw.log++pubsub..bucket.myfoo7/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 to cache LRU end 2024-09-13T23:44:36.465 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.461+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:125 6.10 6:0f4d8598:::pubsub..bucket.myfoo7%2f9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935000 con 0x562e9a6b7000 2024-09-13T23:44:36.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.461+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 41 ==== osd_op_reply(125 pubsub..bucket.myfoo7/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.8 [delete] v35'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 209+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.466 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.461+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:126 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=59b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934c00 con 0x562e9a6b7000 2024-09-13T23:44:36.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.461+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 42 ==== osd_op_reply(126 fud.buckets [call] v35'7 uv7 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.461+0000 7f11410bef40 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-09-13T23:44:36.468 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.461+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:127 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765800 con 0x562e9a764000 2024-09-13T23:44:36.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.465+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 26 ==== osd_op_reply(127 p5YnriCv1nAtykxBrupQ [delete] v35'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.469 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.465+0000 7f11410bef40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-09-13T23:44:36.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.465+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:128 7.15 7:a93a5511:::notify.2:head [notify cookie 94758159994240 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765400 con 0x562e9a6b7400 2024-09-13T23:44:36.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.465+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 33 ==== watch-notify(notify (1) cookie 94758158296064 notify 150323855402 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x562e9a66d1e0 con 0x562e9a6b7400 2024-09-13T23:44:36.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.465+0000 7f113a06c700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855402 cookie 94758158296064 notifier 4830 bl.length()=170 2024-09-13T23:44:36.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.465+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 34 ==== osd_op_reply(128 notify.2 [notify cookie 94758159994240 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.470 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.465+0000 7f113a06c700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:129 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ffc00 con 0x562e9a6b7400 2024-09-13T23:44:36.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.465+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 35 ==== osd_op_reply(129 notify.2 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.465+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 36 ==== watch-notify(notify_complete (2) cookie 94758159994240 notify 150323855402 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562e9a66d380 con 0x562e9a6b7400 2024-09-13T23:44:36.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.465+0000 7f11410bef40 10 removing email index: 2024-09-13T23:44:36.471 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.465+0000 7f11410bef40 10 removing user buckets index 2024-09-13T23:44:36.472 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.465+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:130 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe400 con 0x562e9a6b7000 2024-09-13T23:44:36.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.469+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 43 ==== osd_op_reply(130 fud.buckets [delete] v35'8 uv8 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.469+0000 7f11410bef40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-09-13T23:44:36.473 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.469+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:131 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94758159994240 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a765800 con 0x562e9a764000 2024-09-13T23:44:36.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.469+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 27 ==== watch-notify(notify (1) cookie 94758158300672 notify 150323855382 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x562e9a7cad00 con 0x562e9a764000 2024-09-13T23:44:36.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.469+0000 7f113a06c700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855382 cookie 94758158300672 notifier 4830 bl.length()=160 2024-09-13T23:44:36.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.469+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 28 ==== osd_op_reply(131 notify.6 [notify cookie 94758159994240 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.469+0000 7f113a06c700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:132 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fec00 con 0x562e9a764000 2024-09-13T23:44:36.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.469+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 29 ==== watch-notify(notify_complete (2) cookie 94758159994240 notify 150323855382 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562e9a7caea0 con 0x562e9a764000 2024-09-13T23:44:36.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.469+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 30 ==== osd_op_reply(132 notify.6 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.469+0000 7f11410bef40 10 removing user index: fud 2024-09-13T23:44:36.474 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.469+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:133 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934800 con 0x562e9a764400 2024-09-13T23:44:36.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.473+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 34 ==== osd_op_reply(133 fud [call,call,delete] v35'7 uv7 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.473+0000 7f11410bef40 10 removing default.rgw.meta+users.uid+fud from cache 2024-09-13T23:44:36.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.473+0000 7f11410bef40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-09-13T23:44:36.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.473+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:134 7.3 7:c609908c:::notify.5:head [notify cookie 94758159994240 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935400 con 0x562e9a764000 2024-09-13T23:44:36.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.473+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 31 ==== watch-notify(notify (1) cookie 94758158299520 notify 150323855383 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x562e9a7caea0 con 0x562e9a764000 2024-09-13T23:44:36.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.473+0000 7f113a06c700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855383 cookie 94758158299520 notifier 4830 bl.length()=152 2024-09-13T23:44:36.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.473+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 32 ==== osd_op_reply(134 notify.5 [notify cookie 94758159994240 out=8b] v0'0 uv225 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.477 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.473+0000 7f113a06c700 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:135 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a76e400 con 0x562e9a764000 2024-09-13T23:44:36.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.473+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 33 ==== watch-notify(notify_complete (2) cookie 94758159994240 notify 150323855383 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x562e9a7cb040 con 0x562e9a764000 2024-09-13T23:44:36.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.473+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 34 ==== osd_op_reply(135 notify.5 [notify-ack] v0'0 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.478 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.473+0000 7f1126845700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:44:36.479 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.473+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:136 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94758159990784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935c00 con 0x562e9a6b7000 2024-09-13T23:44:36.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.477+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 44 ==== osd_op_reply(136 data_loggenerations_metadata [watch unwatch cookie 94758159990784] v35'227 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.481 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.477+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:137 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94758158293760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934400 con 0x562e9a6b7400 2024-09-13T23:44:36.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.477+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 37 ==== osd_op_reply(137 notify.0 [watch unwatch cookie 94758158293760] v35'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.477+0000 7f11410bef40 20 remove_watcher() i=0 2024-09-13T23:44:36.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.477+0000 7f11410bef40 2 removed watcher, disabling cache 2024-09-13T23:44:36.484 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.477+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:138 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94758158294912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935800 con 0x562e9a6b7000 2024-09-13T23:44:36.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.481+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.1 v2:172.21.15.167:6800/3875661733 45 ==== osd_op_reply(138 notify.1 [watch unwatch cookie 94758158294912] v35'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a6b7000 2024-09-13T23:44:36.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.481+0000 7f11410bef40 20 remove_watcher() i=1 2024-09-13T23:44:36.486 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.481+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:139 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94758158296064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a935000 con 0x562e9a6b7400 2024-09-13T23:44:36.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.485+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 38 ==== osd_op_reply(139 notify.2 [watch unwatch cookie 94758158296064] v35'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.485+0000 7f11410bef40 20 remove_watcher() i=2 2024-09-13T23:44:36.489 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.485+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:140 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94758158297216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934c00 con 0x562e9a764400 2024-09-13T23:44:36.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.485+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 35 ==== osd_op_reply(140 notify.3 [watch unwatch cookie 94758158297216] v35'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.485+0000 7f11410bef40 20 remove_watcher() i=3 2024-09-13T23:44:36.491 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.485+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:141 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94758158298368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a934800 con 0x562e9a6b7400 2024-09-13T23:44:36.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.489+0000 7f113f877700 1 -- 172.21.15.167:0/42528590 <== osd.0 v2:172.21.15.12:6800/2649399468 39 ==== osd_op_reply(141 notify.4 [watch unwatch cookie 94758158298368] v35'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761200 con 0x562e9a6b7400 2024-09-13T23:44:36.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.489+0000 7f11410bef40 20 remove_watcher() i=4 2024-09-13T23:44:36.494 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.489+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:142 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94758158299520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe800 con 0x562e9a764000 2024-09-13T23:44:36.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.489+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 35 ==== osd_op_reply(142 notify.5 [watch unwatch cookie 94758158299520] v35'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.489+0000 7f11410bef40 20 remove_watcher() i=5 2024-09-13T23:44:36.496 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.489+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:143 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94758158300672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7ff000 con 0x562e9a764000 2024-09-13T23:44:36.498 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.493+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 <== osd.3 v2:172.21.15.167:6816/3688363524 36 ==== osd_op_reply(143 notify.6 [watch unwatch cookie 94758158300672] v35'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a761440 con 0x562e9a764000 2024-09-13T23:44:36.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.493+0000 7f11410bef40 20 remove_watcher() i=6 2024-09-13T23:44:36.499 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.493+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:144 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94758158301824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x562e9a7fe400 con 0x562e9a764400 2024-09-13T23:44:36.501 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f1140078700 1 -- 172.21.15.167:0/42528590 <== osd.2 v2:172.21.15.167:6808/1180369139 36 ==== osd_op_reply(144 notify.7 [watch unwatch cookie 94758158301824] v35'227 uv225 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x562e9a760b40 con 0x562e9a764400 2024-09-13T23:44:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 20 remove_watcher() i=7 2024-09-13T23:44:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562e9a6b7400 msgr2=0x562e9a5e0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562e9a6b7400 0x562e9a5e0100 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562e9a6b7000 msgr2=0x562e9a5dfb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562e9a6b7000 0x562e9a5dfb80 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562e9a764400 msgr2=0x562e9a5e0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562e9a764400 0x562e9a5e0c00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562e9a764000 msgr2=0x562e9a5e0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562e9a764000 0x562e9a5e0680 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a6b6c00 msgr2=0x562e9a5df600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a6b6c00 0x562e9a5df600 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x562e9b0cc030 tx=0x562e9a66ff80 comp rx=0 tx=0).stop 2024-09-13T23:44:36.502 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 msgr2=0x562e9a5deb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/42528590 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9800 0x562e9a5deb00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x562e9a6361b0 tx=0x562e9a3fdda0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 reap_dead start 2024-09-13T23:44:36.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f113f076700 1 -- 172.21.15.167:0/42528590 reap_dead start 2024-09-13T23:44:36.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 shutdown_connections 2024-09-13T23:44:36.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 >> 172.21.15.167:0/42528590 conn(0x562e9a2f9400 msgr2=0x562e9a3f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:36.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 shutdown_connections 2024-09-13T23:44:36.503 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/42528590 wait complete. 2024-09-13T23:44:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562e9a76f000 msgr2=0x562e9a5e1700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x562e9a76f000 0x562e9a5e1700 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562e9a6b7800 msgr2=0x562e9a805080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x562e9a6b7800 0x562e9a805080 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562e9a76e000 msgr2=0x562e9a804580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x562e9a76e000 0x562e9a804580 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562e9a764800 msgr2=0x562e9a804b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x562e9a764800 0x562e9a804b00 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.505 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a7fe000 msgr2=0x562e9a804000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a7fe000 0x562e9a804000 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x562e9a707800 tx=0x562e9a66f3e0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f113f877700 1 -- 172.21.15.167:0/51888767 reap_dead start 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 msgr2=0x562e9a5e1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 0x562e9a5e1180 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x562e9a77d380 tx=0x562e9a77e2a0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 shutdown_connections 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/51888767 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a76ec00 0x562e9a5e1180 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 >> 172.21.15.167:0/51888767 conn(0x562e9a76fc00 msgr2=0x562e9a3fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 shutdown_connections 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/51888767 wait complete. 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/1403426715 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a6b6000 msgr2=0x562e9a5df080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/1403426715 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a6b6000 0x562e9a5df080 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x562e9a637950 tx=0x562e9a66e120 comp rx=0 tx=0).stop 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/1403426715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 msgr2=0x562e9a5de580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:44:36.506 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/1403426715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x562e9a5d9f80 tx=0x562e9a3fd980 comp rx=0 tx=0).stop 2024-09-13T23:44:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/1403426715 shutdown_connections 2024-09-13T23:44:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/1403426715 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x562e9a6b6000 0x562e9a5df080 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 --2- 172.21.15.167:0/1403426715 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x562e9a2f9000 0x562e9a5de580 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:44:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/1403426715 >> 172.21.15.167:0/1403426715 conn(0x562e9a2f8c00 msgr2=0x562e9a3f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:44:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/1403426715 shutdown_connections 2024-09-13T23:44:36.507 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:44:36.497+0000 7f11410bef40 1 -- 172.21.15.167:0/1403426715 wait complete. 2024-09-13T23:45:16.510 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'trim', '--uid', 'foo'] 2024-09-13T23:45:16.511 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'trim', '--uid', 'foo'] 2024-09-13T23:45:16.511 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage trim --uid foo 2024-09-13T23:45:16.557 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:45:16.557 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:45:16.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.550+0000 7efe4799bf40 1 Processor -- start 2024-09-13T23:45:16.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.550+0000 7efe4799bf40 1 -- start start 2024-09-13T23:45:16.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.550+0000 7efe4799bf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5593b6ce1000 0x5593b6fc6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.550+0000 7efe4799bf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5593b6da5500 con 0x5593b6ce1400 2024-09-13T23:45:16.558 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.550+0000 7efe4799bf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5593b6d6ea00 con 0x5593b6ce1000 2024-09-13T23:45:16.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.550+0000 7efe46154700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5593b6ce1000 0x5593b6fc6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.550+0000 7efe45953700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5593b6ce1400 0x5593b6de0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:38752/0 (socket says 172.21.15.167:38752) 2024-09-13T23:45:16.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.550+0000 7efe46154700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5593b6ce1000 0x5593b6fc6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42946/0 (socket says 172.21.15.167:42946) 2024-09-13T23:45:16.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.550+0000 7efe45953700 1 -- 172.21.15.167:0/3409504580 learned_addr learned my addr 172.21.15.167:0/3409504580 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:45:16.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.550+0000 7efe46154700 1 -- 172.21.15.167:0/3409504580 >> v1:172.21.15.12:6789/0 conn(0x5593b6ce1400 legacy=0x5593b6de0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.550+0000 7efe46154700 1 -- 172.21.15.167:0/3409504580 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593b6cf3d40 con 0x5593b6ce1000 2024-09-13T23:45:16.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe46154700 1 --2- 172.21.15.167:0/3409504580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x5593b6fc1a70 tx=0x5593b6de5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a2e9463c794c76 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe45152700 1 -- 172.21.15.167:0/3409504580 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b6d854a0 con 0x5593b6ce1000 2024-09-13T23:45:16.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe45152700 1 -- 172.21.15.167:0/3409504580 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5593b6da5800 con 0x5593b6ce1000 2024-09-13T23:45:16.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe45152700 1 -- 172.21.15.167:0/3409504580 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b6d85600 con 0x5593b6ce1000 2024-09-13T23:45:16.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe4799bf40 1 -- 172.21.15.167:0/3409504580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 msgr2=0x5593b6fc6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe4799bf40 1 --2- 172.21.15.167:0/3409504580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x5593b6fc1a70 tx=0x5593b6de5920 comp rx=0 tx=0).stop 2024-09-13T23:45:16.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe4799bf40 1 -- 172.21.15.167:0/3409504580 shutdown_connections 2024-09-13T23:45:16.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe4799bf40 1 --2- 172.21.15.167:0/3409504580 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe4799bf40 1 -- 172.21.15.167:0/3409504580 >> 172.21.15.167:0/3409504580 conn(0x5593b6ce0c00 msgr2=0x5593b6de0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe4799bf40 1 -- 172.21.15.167:0/3409504580 shutdown_connections 2024-09-13T23:45:16.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe4799bf40 1 -- 172.21.15.167:0/3409504580 wait complete. 2024-09-13T23:45:16.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe4799bf40 1 Processor -- start 2024-09-13T23:45:16.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe4799bf40 1 -- start start 2024-09-13T23:45:16.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe4799bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe4799bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b6d6e500 con 0x5593b6ce1000 2024-09-13T23:45:16.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe46154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe46154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42948/0 (socket says 172.21.15.167:42948) 2024-09-13T23:45:16.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe46154700 1 -- 172.21.15.167:0/3021733232 learned_addr learned my addr 172.21.15.167:0/3021733232 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe46154700 1 -- 172.21.15.167:0/3021733232 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593b6cf3ba0 con 0x5593b6ce1000 2024-09-13T23:45:16.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.554+0000 7efe46154700 1 --2- 172.21.15.167:0/3021733232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x5593b7b45800 tx=0x5593b6de5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6736135881be66d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe44951700 1 -- 172.21.15.167:0/3021733232 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b6d85760 con 0x5593b6ce1000 2024-09-13T23:45:16.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe44951700 1 -- 172.21.15.167:0/3021733232 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5593b6da5b00 con 0x5593b6ce1000 2024-09-13T23:45:16.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe44951700 1 -- 172.21.15.167:0/3021733232 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b6d854a0 con 0x5593b6ce1000 2024-09-13T23:45:16.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 -- 172.21.15.167:0/3021733232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 msgr2=0x5593b6fc6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 --2- 172.21.15.167:0/3021733232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x5593b7b45800 tx=0x5593b6de5800 comp rx=0 tx=0).stop 2024-09-13T23:45:16.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 -- 172.21.15.167:0/3021733232 shutdown_connections 2024-09-13T23:45:16.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 --2- 172.21.15.167:0/3021733232 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 -- 172.21.15.167:0/3021733232 >> 172.21.15.167:0/3021733232 conn(0x5593b6ce0c00 msgr2=0x5593b6de1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 -- 172.21.15.167:0/3021733232 shutdown_connections 2024-09-13T23:45:16.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 -- 172.21.15.167:0/3021733232 wait complete. 2024-09-13T23:45:16.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 Processor -- start 2024-09-13T23:45:16.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 -- start start 2024-09-13T23:45:16.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b6d6e3c0 con 0x5593b6ce1000 2024-09-13T23:45:16.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe46154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe46154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42964/0 (socket says 172.21.15.167:42964) 2024-09-13T23:45:16.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe46154700 1 -- 172.21.15.167:0/3459346417 learned_addr learned my addr 172.21.15.167:0/3459346417 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe46154700 1 -- 172.21.15.167:0/3459346417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593b6cf3a00 con 0x5593b6ce1000 2024-09-13T23:45:16.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe46154700 1 --2- 172.21.15.167:0/3459346417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x5593b7b456b0 tx=0x5593b6de5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4314e700 1 -- 172.21.15.167:0/3459346417 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b6d85600 con 0x5593b6ce1000 2024-09-13T23:45:16.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4314e700 1 -- 172.21.15.167:0/3459346417 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5593b6da5c80 con 0x5593b6ce1000 2024-09-13T23:45:16.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 -- 172.21.15.167:0/3459346417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5593b6cf3860 con 0x5593b6ce1000 2024-09-13T23:45:16.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4314e700 1 -- 172.21.15.167:0/3459346417 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b6d858c0 con 0x5593b6ce1000 2024-09-13T23:45:16.567 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.558+0000 7efe4799bf40 1 -- 172.21.15.167:0/3459346417 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593b6cf36c0 con 0x5593b6ce1000 2024-09-13T23:45:16.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe4314e700 1 -- 172.21.15.167:0/3459346417 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5593b6ddab00 con 0x5593b6ce1000 2024-09-13T23:45:16.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe4799bf40 1 Processor -- start 2024-09-13T23:45:16.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe4314e700 1 --2- 172.21.15.167:0/3459346417 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b7078000 0x5593b6fc7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe4799bf40 1 -- start start 2024-09-13T23:45:16.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe4314e700 1 -- 172.21.15.167:0/3459346417 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5593b6d47180 con 0x5593b6ce1000 2024-09-13T23:45:16.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe46955700 1 --2- 172.21.15.167:0/3459346417 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b7078000 0x5593b6fc7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe4799bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 0x5593b6fc6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe4799bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b6d06640 con 0x5593b6ce1800 2024-09-13T23:45:16.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe46154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 0x5593b6fc6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe46154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 0x5593b6fc6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42982/0 (socket says 172.21.15.167:42982) 2024-09-13T23:45:16.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe46154700 1 -- 172.21.15.167:0/2054446493 learned_addr learned my addr 172.21.15.167:0/2054446493 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe46955700 1 --2- 172.21.15.167:0/3459346417 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b7078000 0x5593b6fc7080 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x5593b7b45f20 tx=0x5593b7040b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe46154700 1 -- 172.21.15.167:0/2054446493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593b6cf2000 con 0x5593b6ce1800 2024-09-13T23:45:16.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe46154700 1 --2- 172.21.15.167:0/2054446493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 0x5593b6fc6b00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x5593b7b54f60 tx=0x5593b7040c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76017d6ddf9f4d16 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe40949700 1 -- 172.21.15.167:0/2054446493 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b6d85600 con 0x5593b6ce1800 2024-09-13T23:45:16.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe40949700 1 -- 172.21.15.167:0/2054446493 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5593b6da5c80 con 0x5593b6ce1800 2024-09-13T23:45:16.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.562+0000 7efe40949700 1 -- 172.21.15.167:0/2054446493 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b6d858c0 con 0x5593b6ce1800 2024-09-13T23:45:16.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe4799bf40 1 -- 172.21.15.167:0/2054446493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 msgr2=0x5593b6fc6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe4799bf40 1 --2- 172.21.15.167:0/2054446493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 0x5593b6fc6b00 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x5593b7b54f60 tx=0x5593b7040c60 comp rx=0 tx=0).stop 2024-09-13T23:45:16.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe4799bf40 1 -- 172.21.15.167:0/2054446493 shutdown_connections 2024-09-13T23:45:16.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe4799bf40 1 --2- 172.21.15.167:0/2054446493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 0x5593b6fc6b00 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe4799bf40 1 -- 172.21.15.167:0/2054446493 >> 172.21.15.167:0/2054446493 conn(0x5593b6ce1400 msgr2=0x5593b6de1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe4799bf40 1 -- 172.21.15.167:0/2054446493 shutdown_connections 2024-09-13T23:45:16.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe4799bf40 1 -- 172.21.15.167:0/2054446493 wait complete. 2024-09-13T23:45:16.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe4799bf40 1 Processor -- start 2024-09-13T23:45:16.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe4799bf40 1 -- start start 2024-09-13T23:45:16.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe4799bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 0x5593b6fc6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe4799bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b6d068c0 con 0x5593b6ce1800 2024-09-13T23:45:16.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe46154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 0x5593b6fc6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe46154700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 0x5593b6fc6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42984/0 (socket says 172.21.15.167:42984) 2024-09-13T23:45:16.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 learned_addr learned my addr 172.21.15.167:0/2022159176 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593b6cf2ea0 con 0x5593b6ce1800 2024-09-13T23:45:16.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe46154700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 0x5593b6fc6b00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x5593b707a5a0 tx=0x5593b7040e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe3f146700 1 -- 172.21.15.167:0/2022159176 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b6d85760 con 0x5593b6ce1800 2024-09-13T23:45:16.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.566+0000 7efe3f146700 1 -- 172.21.15.167:0/2022159176 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5593b70a6000 con 0x5593b6ce1800 2024-09-13T23:45:16.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe3f146700 1 -- 172.21.15.167:0/2022159176 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b6d854a0 con 0x5593b6ce1800 2024-09-13T23:45:16.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5593b6cf2820 con 0x5593b6ce1800 2024-09-13T23:45:16.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593b6cf29c0 con 0x5593b6ce1800 2024-09-13T23:45:16.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe3f146700 1 -- 172.21.15.167:0/2022159176 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5593b6ddadc0 con 0x5593b6ce1800 2024-09-13T23:45:16.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe3f146700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b7079000 0x5593b6fc7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe3f146700 1 -- 172.21.15.167:0/2022159176 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5593b6d47340 con 0x5593b6ce1800 2024-09-13T23:45:16.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe45953700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b7079000 0x5593b6fc7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe4799bf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe4799bf40 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5593b7079400 0x5593b6fc7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7079800 con 0x5593b7079400 2024-09-13T23:45:16.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe46955700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5593b7079400 0x5593b6fc7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe45953700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b7079000 0x5593b6fc7600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5593b707bef0 tx=0x5593b7110000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe46955700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5593b7079400 0x5593b6fc7b80 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe4799bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe4799bf40 20 realm 2024-09-13T23:45:16.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe4799bf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.570+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7079c00 con 0x5593b7079400 2024-09-13T23:45:16.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:45:16.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7079800 con 0x5593b7079400 2024-09-13T23:45:16.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5593b7079c00 0x5593b6fc8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118000 con 0x5593b7079c00 2024-09-13T23:45:16.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe46154700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5593b7079c00 0x5593b6fc8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe46154700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5593b7079c00 0x5593b6fc8100 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:45:16.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079c00 2024-09-13T23:45:16.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:45:16.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 20 searching for the correct realm 2024-09-13T23:45:16.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5593b7118800 0x5593b6fc8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118c00 con 0x5593b7118800 2024-09-13T23:45:16.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.574+0000 7efe45953700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5593b7118800 0x5593b6fc8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe45953700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5593b7118800 0x5593b6fc8680 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118c00 con 0x5593b7079c00 2024-09-13T23:45:16.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe46154700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5593b7118000 0x5593b6fc8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118000 2024-09-13T23:45:16.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe46955700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5593b7118000 0x5593b6fc8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe46955700 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5593b7118000 0x5593b6fc8c00 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079400 2024-09-13T23:45:16.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079400 2024-09-13T23:45:16.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.578+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118800 2024-09-13T23:45:16.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079400 2024-09-13T23:45:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079c00 2024-09-13T23:45:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079c00 2024-09-13T23:45:16.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079400 2024-09-13T23:45:16.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118000 2024-09-13T23:45:16.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079400 2024-09-13T23:45:16.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.582+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118000 2024-09-13T23:45:16.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079c00 2024-09-13T23:45:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118800 2024-09-13T23:45:16.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079400 2024-09-13T23:45:16.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079c00 2024-09-13T23:45:16.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118800 2024-09-13T23:45:16.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079400 2024-09-13T23:45:16.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.586+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118000 2024-09-13T23:45:16.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079400 2024-09-13T23:45:16.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118800 2024-09-13T23:45:16.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118000 2024-09-13T23:45:16.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079400 2024-09-13T23:45:16.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118000 2024-09-13T23:45:16.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118800 2024-09-13T23:45:16.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.590+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118000 2024-09-13T23:45:16.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.594+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.594+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079c00 2024-09-13T23:45:16.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.594+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.600 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.594+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079c00 2024-09-13T23:45:16.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.594+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.594+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118800 2024-09-13T23:45:16.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.594+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.601 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.594+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079c00 2024-09-13T23:45:16.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.594+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.594+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079c00 2024-09-13T23:45:16.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:45:16.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:45:16.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:45:16.603 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:45:16.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119000 con 0x5593b7079400 2024-09-13T23:45:16.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.604 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119400 con 0x5593b7079c00 2024-09-13T23:45:16.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:45:16.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119800 con 0x5593b7079c00 2024-09-13T23:45:16.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:45:16.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 zone default found 2024-09-13T23:45:16.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 4 Realm: () 2024-09-13T23:45:16.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:45:16.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:45:16.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:45:16.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 zonegroup default 2024-09-13T23:45:16.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.606 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119c00 con 0x5593b7118000 2024-09-13T23:45:16.607 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119000 con 0x5593b7118800 2024-09-13T23:45:16.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.598+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119c00 con 0x5593b7118800 2024-09-13T23:45:16.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.602+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.602+0000 7efe4799bf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.602+0000 7efe4799bf40 20 started sync module instance, tier type = 2024-09-13T23:45:16.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.602+0000 7efe4799bf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:45:16.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.602+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7079800 con 0x5593b7079c00 2024-09-13T23:45:16.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.602+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.602+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94092918512384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118c00 con 0x5593b7079c00 2024-09-13T23:45:16.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.602+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7079400 2024-09-13T23:45:16.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.606+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.606+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94092918513536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119400 con 0x5593b7079400 2024-09-13T23:45:16.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.606+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94092918512384] v35'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.606+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119800 con 0x5593b7079c00 2024-09-13T23:45:16.616 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.610+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94092918513536] v35'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.610+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.610+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94092918514688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119000 con 0x5593b7079c00 2024-09-13T23:45:16.617 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.610+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119c00 con 0x5593b7118000 2024-09-13T23:45:16.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.614+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94092918514688] v35'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.614+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.614+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94092918515840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7078800 con 0x5593b7118000 2024-09-13T23:45:16.620 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.614+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7079800 con 0x5593b7079c00 2024-09-13T23:45:16.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.614+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94092918515840] v35'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.614+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.622 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.614+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94092918516992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7078800 con 0x5593b7079c00 2024-09-13T23:45:16.623 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.614+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118800 2024-09-13T23:45:16.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.618+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94092918516992] v35'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.618+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.618+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94092918518144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119400 con 0x5593b7118800 2024-09-13T23:45:16.625 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.618+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119c00 con 0x5593b7118800 2024-09-13T23:45:16.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.618+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94092918518144] v35'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94092918519296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7118400 con 0x5593b7118800 2024-09-13T23:45:16.627 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119400 con 0x5593b7118000 2024-09-13T23:45:16.629 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94092918519296] v35'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'228 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94092918520448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119c00 con 0x5593b7118000 2024-09-13T23:45:16.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe4799bf40 20 add_watcher() i=0 2024-09-13T23:45:16.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe4799bf40 20 add_watcher() i=1 2024-09-13T23:45:16.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe4799bf40 20 add_watcher() i=2 2024-09-13T23:45:16.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe4799bf40 20 add_watcher() i=3 2024-09-13T23:45:16.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe4799bf40 20 add_watcher() i=4 2024-09-13T23:45:16.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe4799bf40 20 add_watcher() i=5 2024-09-13T23:45:16.630 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.622+0000 7efe4799bf40 20 add_watcher() i=6 2024-09-13T23:45:16.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.626+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94092918520448] v35'229 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.631 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.626+0000 7efe4799bf40 20 add_watcher() i=7 2024-09-13T23:45:16.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.626+0000 7efe4799bf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:45:16.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.626+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7126000 con 0x5593b7079400 2024-09-13T23:45:16.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.626+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.632 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.626+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94092919947264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7126400 con 0x5593b7079400 2024-09-13T23:45:16.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.626+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94092919947264] v35'228 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.626+0000 7efe2d122700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:45:16.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.626+0000 7efe4799bf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:45:16.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.626+0000 7efe4799bf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:45:16.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe4799bf40 1 Processor -- start 2024-09-13T23:45:16.635 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe4799bf40 1 -- start start 2024-09-13T23:45:16.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe4799bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 0x5593b6fc9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe4799bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b6d6e3c0 con 0x5593b7126c00 2024-09-13T23:45:16.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe45953700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 0x5593b6fc9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe45953700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 0x5593b6fc9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42994/0 (socket says 172.21.15.167:42994) 2024-09-13T23:45:16.636 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe45953700 1 -- 172.21.15.167:0/1306926464 learned_addr learned my addr 172.21.15.167:0/1306926464 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe45953700 1 -- 172.21.15.167:0/1306926464 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593b6cf2b60 con 0x5593b7126c00 2024-09-13T23:45:16.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe45953700 1 --2- 172.21.15.167:0/1306926464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 0x5593b6fc9180 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x5593b7132090 tx=0x5593b7041f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2922773e06fbdf6a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe2c921700 1 -- 172.21.15.167:0/1306926464 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b7128160 con 0x5593b7126c00 2024-09-13T23:45:16.637 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe2c921700 1 -- 172.21.15.167:0/1306926464 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5593b70a6a80 con 0x5593b7126c00 2024-09-13T23:45:16.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe2c921700 1 -- 172.21.15.167:0/1306926464 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b71282c0 con 0x5593b7126c00 2024-09-13T23:45:16.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe4799bf40 1 -- 172.21.15.167:0/1306926464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 msgr2=0x5593b6fc9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.638 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe4799bf40 1 --2- 172.21.15.167:0/1306926464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 0x5593b6fc9180 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x5593b7132090 tx=0x5593b7041f20 comp rx=0 tx=0).stop 2024-09-13T23:45:16.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe4799bf40 1 -- 172.21.15.167:0/1306926464 shutdown_connections 2024-09-13T23:45:16.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe4799bf40 1 --2- 172.21.15.167:0/1306926464 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 0x5593b6fc9180 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe4799bf40 1 -- 172.21.15.167:0/1306926464 >> 172.21.15.167:0/1306926464 conn(0x5593b7126800 msgr2=0x5593b6de2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe4799bf40 1 -- 172.21.15.167:0/1306926464 shutdown_connections 2024-09-13T23:45:16.639 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.630+0000 7efe4799bf40 1 -- 172.21.15.167:0/1306926464 wait complete. 2024-09-13T23:45:16.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe4799bf40 1 Processor -- start 2024-09-13T23:45:16.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe4799bf40 1 -- start start 2024-09-13T23:45:16.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe4799bf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 0x5593b6fc9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe4799bf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593b6d6e500 con 0x5593b7126c00 2024-09-13T23:45:16.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe45953700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 0x5593b6fc9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe45953700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 0x5593b6fc9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:42998/0 (socket says 172.21.15.167:42998) 2024-09-13T23:45:16.640 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe45953700 1 -- 172.21.15.167:0/1418628475 learned_addr learned my addr 172.21.15.167:0/1418628475 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe45953700 1 -- 172.21.15.167:0/1418628475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593b6cf2d00 con 0x5593b7126c00 2024-09-13T23:45:16.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe45953700 1 --2- 172.21.15.167:0/1418628475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 0x5593b6fc9180 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x5593b7133350 tx=0x5593b71100c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe2b11e700 1 -- 172.21.15.167:0/1418628475 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b7128420 con 0x5593b7126c00 2024-09-13T23:45:16.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe2b11e700 1 -- 172.21.15.167:0/1418628475 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5593b70a6780 con 0x5593b7126c00 2024-09-13T23:45:16.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe4799bf40 1 -- 172.21.15.167:0/1418628475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5593b6cf2340 con 0x5593b7126c00 2024-09-13T23:45:16.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe2b11e700 1 -- 172.21.15.167:0/1418628475 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5593b7128160 con 0x5593b7126c00 2024-09-13T23:45:16.642 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe4799bf40 1 -- 172.21.15.167:0/1418628475 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593b6cf2b60 con 0x5593b7126c00 2024-09-13T23:45:16.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.634+0000 7efe2b11e700 1 -- 172.21.15.167:0/1418628475 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5593b6ddb8c0 con 0x5593b7126c00 2024-09-13T23:45:16.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe2b11e700 1 --2- 172.21.15.167:0/1418628475 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b71b6000 0x5593b71bc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe2b11e700 1 -- 172.21.15.167:0/1418628475 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5593b6d476c0 con 0x5593b7126c00 2024-09-13T23:45:16.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe4799bf40 5 note: GC not initialized 2024-09-13T23:45:16.643 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe46955700 1 --2- 172.21.15.167:0/1418628475 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b71b6000 0x5593b71bc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.644 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe46955700 1 --2- 172.21.15.167:0/1418628475 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b71b6000 0x5593b71bc000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x5593b70bfc20 tx=0x5593b7040a80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe28919700 20 reqs_thread_entry: start 2024-09-13T23:45:16.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe4799bf40 20 init_complete bucket index max shards: 11 2024-09-13T23:45:16.645 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe4799bf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:45:16.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe26915700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:45:46 2024 2024-09-13T23:45:16.646 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:45:16.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe260f3700 20 reqs_thread_entry: start 2024-09-13T23:45:16.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe4799bf40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:45:16.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe4799bf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7127800 con 0x5593b7079400 2024-09-13T23:45:16.646 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe4799bf40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:45:16.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe4799bf40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:45:16.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe4799bf40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:45:16.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe4799bf40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:45:16.647 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.638+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7127c00 con 0x5593b7118800 2024-09-13T23:45:16.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.642+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 usage.10 [call] v35'32 uv32 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.649 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.642+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:64 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b71b6400 con 0x5593b7118800 2024-09-13T23:45:16.651 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.642+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(64 usage.10 [call] v35'33 uv32 ondisk = -61 ((61) No data available)) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.646+0000 7efe2d122700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:45:16.652 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.646+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94092919947264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b71b6800 con 0x5593b7079400 2024-09-13T23:45:16.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.646+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94092919947264] v35'229 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.655 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.646+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94092918512384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b71b6c00 con 0x5593b7079c00 2024-09-13T23:45:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.650+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94092918512384] v35'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.650+0000 7efe4799bf40 20 remove_watcher() i=0 2024-09-13T23:45:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.650+0000 7efe4799bf40 2 removed watcher, disabling cache 2024-09-13T23:45:16.657 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.650+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94092918513536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7126000 con 0x5593b7079400 2024-09-13T23:45:16.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.654+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94092918513536] v35'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7079400 2024-09-13T23:45:16.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.654+0000 7efe4799bf40 20 remove_watcher() i=1 2024-09-13T23:45:16.659 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.654+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94092918514688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7126400 con 0x5593b7079c00 2024-09-13T23:45:16.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.654+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94092918514688] v35'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.654+0000 7efe4799bf40 20 remove_watcher() i=2 2024-09-13T23:45:16.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.654+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94092918515840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7127800 con 0x5593b7118000 2024-09-13T23:45:16.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.658+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94092918515840] v35'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.658+0000 7efe4799bf40 20 remove_watcher() i=3 2024-09-13T23:45:16.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.658+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94092918516992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119400 con 0x5593b7079c00 2024-09-13T23:45:16.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.658+0000 7efe46154700 1 -- 172.21.15.167:0/2022159176 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94092918516992] v35'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117200 con 0x5593b7079c00 2024-09-13T23:45:16.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.658+0000 7efe4799bf40 20 remove_watcher() i=4 2024-09-13T23:45:16.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.658+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94092918518144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b7119c00 con 0x5593b7118800 2024-09-13T23:45:16.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.662+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94092918518144] v35'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.662+0000 7efe4799bf40 20 remove_watcher() i=5 2024-09-13T23:45:16.669 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.662+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94092918519296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b71b6c00 con 0x5593b7118800 2024-09-13T23:45:16.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.662+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94092918519296] v35'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7117440 con 0x5593b7118800 2024-09-13T23:45:16.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 20 remove_watcher() i=6 2024-09-13T23:45:16.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94092918520448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5593b71b7c00 con 0x5593b7118000 2024-09-13T23:45:16.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe46955700 1 -- 172.21.15.167:0/2022159176 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94092918520448] v35'230 uv228 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5593b7116b40 con 0x5593b7118000 2024-09-13T23:45:16.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 20 remove_watcher() i=7 2024-09-13T23:45:16.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5593b7079c00 msgr2=0x5593b6fc8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5593b7079c00 0x5593b6fc8100 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5593b7079400 msgr2=0x5593b6fc7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5593b7079400 0x5593b6fc7b80 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5593b7118000 msgr2=0x5593b6fc8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5593b7118000 0x5593b6fc8c00 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5593b7118800 msgr2=0x5593b6fc8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5593b7118800 0x5593b6fc8680 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b7079000 msgr2=0x5593b6fc7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b7079000 0x5593b6fc7600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x5593b707bef0 tx=0x5593b7110000 comp rx=0 tx=0).stop 2024-09-13T23:45:16.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe45953700 1 -- 172.21.15.167:0/2022159176 reap_dead start 2024-09-13T23:45:16.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 msgr2=0x5593b6fc6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 0x5593b6fc6b00 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x5593b707a5a0 tx=0x5593b7040e40 comp rx=0 tx=0).stop 2024-09-13T23:45:16.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 shutdown_connections 2024-09-13T23:45:16.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 --2- 172.21.15.167:0/2022159176 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1800 0x5593b6fc6b00 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 >> 172.21.15.167:0/2022159176 conn(0x5593b6ce1400 msgr2=0x5593b6de1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 shutdown_connections 2024-09-13T23:45:16.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/2022159176 wait complete. 2024-09-13T23:45:16.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/1418628475 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b71b6000 msgr2=0x5593b71bc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 --2- 172.21.15.167:0/1418628475 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b71b6000 0x5593b71bc000 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x5593b70bfc20 tx=0x5593b7040a80 comp rx=0 tx=0).stop 2024-09-13T23:45:16.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/1418628475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 msgr2=0x5593b6fc9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 --2- 172.21.15.167:0/1418628475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 0x5593b6fc9180 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x5593b7133350 tx=0x5593b71100c0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/1418628475 shutdown_connections 2024-09-13T23:45:16.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 --2- 172.21.15.167:0/1418628475 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b71b6000 0x5593b71bc000 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 --2- 172.21.15.167:0/1418628475 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b7126c00 0x5593b6fc9180 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.666+0000 7efe4799bf40 1 -- 172.21.15.167:0/1418628475 >> 172.21.15.167:0/1418628475 conn(0x5593b7126800 msgr2=0x5593b6de2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.670+0000 7efe4799bf40 1 -- 172.21.15.167:0/1418628475 shutdown_connections 2024-09-13T23:45:16.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.670+0000 7efe4799bf40 1 -- 172.21.15.167:0/1418628475 wait complete. 2024-09-13T23:45:16.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.670+0000 7efe4799bf40 1 -- 172.21.15.167:0/3459346417 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b7078000 msgr2=0x5593b6fc7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.670+0000 7efe4799bf40 1 --2- 172.21.15.167:0/3459346417 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b7078000 0x5593b6fc7080 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x5593b7b45f20 tx=0x5593b7040b40 comp rx=0 tx=0).stop 2024-09-13T23:45:16.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.670+0000 7efe4799bf40 1 -- 172.21.15.167:0/3459346417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 msgr2=0x5593b6fc6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.670+0000 7efe4799bf40 1 --2- 172.21.15.167:0/3459346417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x5593b7b456b0 tx=0x5593b6de5920 comp rx=0 tx=0).stop 2024-09-13T23:45:16.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.670+0000 7efe4799bf40 1 -- 172.21.15.167:0/3459346417 shutdown_connections 2024-09-13T23:45:16.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.670+0000 7efe4799bf40 1 --2- 172.21.15.167:0/3459346417 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5593b7078000 0x5593b6fc7080 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.670+0000 7efe4799bf40 1 --2- 172.21.15.167:0/3459346417 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5593b6ce1000 0x5593b6fc6580 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.670+0000 7efe4799bf40 1 -- 172.21.15.167:0/3459346417 >> 172.21.15.167:0/3459346417 conn(0x5593b6ce0c00 msgr2=0x5593b6de1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.670+0000 7efe4799bf40 1 -- 172.21.15.167:0/3459346417 shutdown_connections 2024-09-13T23:45:16.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.670+0000 7efe4799bf40 1 -- 172.21.15.167:0/3459346417 wait complete. 2024-09-13T23:45:16.680 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-09-13T23:45:16.680 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-09-13T23:45:16.681 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-09-13T23:45:16.722 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:45:16.722 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:45:16.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.714+0000 7f56cfcfaf40 1 Processor -- start 2024-09-13T23:45:16.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.714+0000 7f56cfcfaf40 1 -- start start 2024-09-13T23:45:16.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.714+0000 7f56cfcfaf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f66d873400 0x55f66db58580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cfcfaf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f66d937500 con 0x55f66d873000 2024-09-13T23:45:16.723 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cfcfaf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f66d900a00 con 0x55f66d873400 2024-09-13T23:45:16.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cdcb2700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f66d873400 0x55f66db58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56ce4b3700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55f66d873000 0x55f66d972800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:38764/0 (socket says 172.21.15.167:38764) 2024-09-13T23:45:16.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1014836423 learned_addr learned my addr 172.21.15.167:0/1014836423 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:45:16.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1014836423 >> v1:172.21.15.12:6789/0 conn(0x55f66d873000 legacy=0x55f66d972800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.724 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1014836423 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f66d885d40 con 0x55f66d873400 2024-09-13T23:45:16.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cdcb2700 1 --2- 172.21.15.167:0/1014836423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873400 0x55f66db58580 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55f66db53920 tx=0x55f66d977920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6774ac989c0d9ff2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.725 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cd4b1700 1 -- 172.21.15.167:0/1014836423 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66d9174a0 con 0x55f66d873400 2024-09-13T23:45:16.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cd4b1700 1 -- 172.21.15.167:0/1014836423 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f66d937800 con 0x55f66d873400 2024-09-13T23:45:16.726 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cd4b1700 1 -- 172.21.15.167:0/1014836423 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66d917600 con 0x55f66d873400 2024-09-13T23:45:16.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1014836423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873400 msgr2=0x55f66db58580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1014836423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873400 0x55f66db58580 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x55f66db53920 tx=0x55f66d977920 comp rx=0 tx=0).stop 2024-09-13T23:45:16.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1014836423 shutdown_connections 2024-09-13T23:45:16.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1014836423 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873400 0x55f66db58580 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1014836423 >> 172.21.15.167:0/1014836423 conn(0x55f66d872c00 msgr2=0x55f66d972000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1014836423 shutdown_connections 2024-09-13T23:45:16.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1014836423 wait complete. 2024-09-13T23:45:16.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cfcfaf40 1 Processor -- start 2024-09-13T23:45:16.727 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.718+0000 7f56cfcfaf40 1 -- start start 2024-09-13T23:45:16.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 0x55f66db58580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d900500 con 0x55f66d873000 2024-09-13T23:45:16.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56ce4b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 0x55f66db58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56ce4b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 0x55f66db58580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43024/0 (socket says 172.21.15.167:43024) 2024-09-13T23:45:16.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56ce4b3700 1 -- 172.21.15.167:0/3431986857 learned_addr learned my addr 172.21.15.167:0/3431986857 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56ce4b3700 1 -- 172.21.15.167:0/3431986857 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f66d885ba0 con 0x55f66d873000 2024-09-13T23:45:16.729 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56ce4b3700 1 --2- 172.21.15.167:0/3431986857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 0x55f66db58580 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55f66d9cfe60 tx=0x55f66d977b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa495e71e499000 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cccb0700 1 -- 172.21.15.167:0/3431986857 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66d9174a0 con 0x55f66d873000 2024-09-13T23:45:16.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cccb0700 1 -- 172.21.15.167:0/3431986857 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f66d937b00 con 0x55f66d873000 2024-09-13T23:45:16.730 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cccb0700 1 -- 172.21.15.167:0/3431986857 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66d917600 con 0x55f66d873000 2024-09-13T23:45:16.735 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/3431986857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 msgr2=0x55f66db58580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/3431986857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 0x55f66db58580 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55f66d9cfe60 tx=0x55f66d977b60 comp rx=0 tx=0).stop 2024-09-13T23:45:16.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/3431986857 shutdown_connections 2024-09-13T23:45:16.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/3431986857 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 0x55f66db58580 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/3431986857 >> 172.21.15.167:0/3431986857 conn(0x55f66d872c00 msgr2=0x55f66d973000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/3431986857 shutdown_connections 2024-09-13T23:45:16.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/3431986857 wait complete. 2024-09-13T23:45:16.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 Processor -- start 2024-09-13T23:45:16.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 -- start start 2024-09-13T23:45:16.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 0x55f66db58580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.736 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56cfcfaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d9003c0 con 0x55f66d873000 2024-09-13T23:45:16.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.722+0000 7f56ce4b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 0x55f66db58580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56ce4b3700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 0x55f66db58580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43030/0 (socket says 172.21.15.167:43030) 2024-09-13T23:45:16.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56ce4b3700 1 -- 172.21.15.167:0/2880621218 learned_addr learned my addr 172.21.15.167:0/2880621218 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56ce4b3700 1 -- 172.21.15.167:0/2880621218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f66d885a00 con 0x55f66d873000 2024-09-13T23:45:16.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56ce4b3700 1 --2- 172.21.15.167:0/2880621218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 0x55f66db58580 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x55f66d9cfec0 tx=0x55f66d977a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cb4ad700 1 -- 172.21.15.167:0/2880621218 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66d9178c0 con 0x55f66d873000 2024-09-13T23:45:16.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cb4ad700 1 -- 172.21.15.167:0/2880621218 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f66d937c80 con 0x55f66d873000 2024-09-13T23:45:16.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/2880621218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f66d885860 con 0x55f66d873000 2024-09-13T23:45:16.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cb4ad700 1 -- 172.21.15.167:0/2880621218 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66d9174a0 con 0x55f66d873000 2024-09-13T23:45:16.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/2880621218 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f66d8856c0 con 0x55f66d873000 2024-09-13T23:45:16.737 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cfcfaf40 1 Processor -- start 2024-09-13T23:45:16.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cfcfaf40 1 -- start start 2024-09-13T23:45:16.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cfcfaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 0x55f66db58b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cfcfaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d900000 con 0x55f66d873800 2024-09-13T23:45:16.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cb4ad700 1 -- 172.21.15.167:0/2880621218 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f66d96cb00 con 0x55f66d873000 2024-09-13T23:45:16.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cecb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 0x55f66db58b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cecb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 0x55f66db58b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43036/0 (socket says 172.21.15.167:43036) 2024-09-13T23:45:16.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cecb4700 1 -- 172.21.15.167:0/1321491457 learned_addr learned my addr 172.21.15.167:0/1321491457 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.726+0000 7f56cb4ad700 1 --2- 172.21.15.167:0/2880621218 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dc1e000 0x55f66db59080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cb4ad700 1 -- 172.21.15.167:0/2880621218 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f66d8d9180 con 0x55f66d873000 2024-09-13T23:45:16.738 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56ce4b3700 1 --2- 172.21.15.167:0/2880621218 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dc1e000 0x55f66db59080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cecb4700 1 -- 172.21.15.167:0/1321491457 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f66d884000 con 0x55f66d873800 2024-09-13T23:45:16.739 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cecb4700 1 --2- 172.21.15.167:0/1321491457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 0x55f66db58b00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x55f66dbfe510 tx=0x55f66dbd2de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde651f18085b869 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56c8ca8700 1 -- 172.21.15.167:0/1321491457 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66d9178c0 con 0x55f66d873800 2024-09-13T23:45:16.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56ce4b3700 1 --2- 172.21.15.167:0/2880621218 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dc1e000 0x55f66db59080 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55f66e6eb5f0 tx=0x55f66dbd2240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56c8ca8700 1 -- 172.21.15.167:0/1321491457 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f66dc20000 con 0x55f66d873800 2024-09-13T23:45:16.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56c8ca8700 1 -- 172.21.15.167:0/1321491457 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66d9174a0 con 0x55f66d873800 2024-09-13T23:45:16.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1321491457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 msgr2=0x55f66db58b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1321491457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 0x55f66db58b00 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x55f66dbfe510 tx=0x55f66dbd2de0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1321491457 shutdown_connections 2024-09-13T23:45:16.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1321491457 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 0x55f66db58b00 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.742 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1321491457 >> 172.21.15.167:0/1321491457 conn(0x55f66d873400 msgr2=0x55f66d973800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1321491457 shutdown_connections 2024-09-13T23:45:16.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1321491457 wait complete. 2024-09-13T23:45:16.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cfcfaf40 1 Processor -- start 2024-09-13T23:45:16.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cfcfaf40 1 -- start start 2024-09-13T23:45:16.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cfcfaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 0x55f66db58b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cfcfaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d898640 con 0x55f66d873800 2024-09-13T23:45:16.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cecb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 0x55f66db58b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cecb4700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 0x55f66db58b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43040/0 (socket says 172.21.15.167:43040) 2024-09-13T23:45:16.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 learned_addr learned my addr 172.21.15.167:0/1740297274 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.743 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.730+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f66d884ea0 con 0x55f66d873800 2024-09-13T23:45:16.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56cecb4700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 0x55f66db58b00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x55f66e6ea420 tx=0x55f66dbd2840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56c74a5700 1 -- 172.21.15.167:0/1740297274 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66d917b80 con 0x55f66d873800 2024-09-13T23:45:16.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56c74a5700 1 -- 172.21.15.167:0/1740297274 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f66dc20180 con 0x55f66d873800 2024-09-13T23:45:16.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f66d884820 con 0x55f66d873800 2024-09-13T23:45:16.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56c74a5700 1 -- 172.21.15.167:0/1740297274 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66d9178c0 con 0x55f66d873800 2024-09-13T23:45:16.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f66d8849c0 con 0x55f66d873800 2024-09-13T23:45:16.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56cfcfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56c74a5700 1 -- 172.21.15.167:0/1740297274 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f66d96cdc0 con 0x55f66d873800 2024-09-13T23:45:16.744 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56c74a5700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dc1ec00 0x55f66db59600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.745 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56c74a5700 1 -- 172.21.15.167:0/1740297274 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f66d8d9340 con 0x55f66d873800 2024-09-13T23:45:16.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f66dc1f000 0x55f66db59b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.746 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56cdcb2700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dc1ec00 0x55f66db59600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f400 con 0x55f66dc1f000 2024-09-13T23:45:16.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.734+0000 7f56cecb4700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f66dc1f000 0x55f66db59b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cdcb2700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dc1ec00 0x55f66db59600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55f66db53920 tx=0x55f66d977800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cecb4700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f66dc1f000 0x55f66db59b80 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 20 realm 2024-09-13T23:45:16.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f000 2024-09-13T23:45:16.747 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:45:16.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1fc00 con 0x55f66dc1f000 2024-09-13T23:45:16.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f66dc1f400 0x55f66db5a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f400 2024-09-13T23:45:16.748 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56ce4b3700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f66dc1f400 0x55f66db5a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.749 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56ce4b3700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f66dc1f400 0x55f66db5a100 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.750 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:45:16.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.738+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1fc00 con 0x55f66dc1f400 2024-09-13T23:45:16.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56cfcfaf40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:45:16.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56cfcfaf40 20 searching for the correct realm 2024-09-13T23:45:16.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f66dbb0000 0x55f66db5a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb0400 con 0x55f66dbb0000 2024-09-13T23:45:16.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56cdcb2700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f66dbb0000 0x55f66db5a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56cdcb2700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f66dbb0000 0x55f66db5a680 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.751 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb0400 con 0x55f66dc1f400 2024-09-13T23:45:16.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56ce4b3700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f66dbb0400 0x55f66db5ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0400 2024-09-13T23:45:16.752 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.742+0000 7f56cecb4700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f66dbb0400 0x55f66db5ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.746+0000 7f56cecb4700 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f66dbb0400 0x55f66db5ac00 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.746+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.746+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f000 2024-09-13T23:45:16.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.746+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.746+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f000 2024-09-13T23:45:16.753 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.746+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.746+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0000 2024-09-13T23:45:16.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.746+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.746+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f000 2024-09-13T23:45:16.754 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.746+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.746+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f400 2024-09-13T23:45:16.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.746+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.755 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.750+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f400 2024-09-13T23:45:16.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.750+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.750+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f000 2024-09-13T23:45:16.756 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.750+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.750+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0400 2024-09-13T23:45:16.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.750+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.757 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.750+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f000 2024-09-13T23:45:16.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.750+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.750+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0400 2024-09-13T23:45:16.758 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.750+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.750+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f400 2024-09-13T23:45:16.759 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.754+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.754+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0000 2024-09-13T23:45:16.760 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.754+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.754+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f000 2024-09-13T23:45:16.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.754+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.754+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f400 2024-09-13T23:45:16.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.754+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.761 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.754+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0000 2024-09-13T23:45:16.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.754+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.754+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f000 2024-09-13T23:45:16.762 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.754+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.754+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0400 2024-09-13T23:45:16.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.758+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.763 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.758+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f000 2024-09-13T23:45:16.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.758+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.758+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0000 2024-09-13T23:45:16.764 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.758+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.758+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0400 2024-09-13T23:45:16.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.758+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.765 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.758+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f000 2024-09-13T23:45:16.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.758+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.766 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.758+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0400 2024-09-13T23:45:16.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.758+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.758+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0000 2024-09-13T23:45:16.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.762+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.767 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.762+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0400 2024-09-13T23:45:16.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.762+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.768 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.762+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f400 2024-09-13T23:45:16.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.762+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.769 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.762+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f400 2024-09-13T23:45:16.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.762+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.762+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0000 2024-09-13T23:45:16.770 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.762+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.771 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.762+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f400 2024-09-13T23:45:16.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f400 2024-09-13T23:45:16.772 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cfcfaf40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:45:16.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cfcfaf40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:45:16.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cfcfaf40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:45:16.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cfcfaf40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:45:16.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cfcfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.773 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb0800 con 0x55f66dc1f000 2024-09-13T23:45:16.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cfcfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cfcfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb0c00 con 0x55f66dc1f400 2024-09-13T23:45:16.774 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cfcfaf40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:45:16.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cfcfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.766+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb1000 con 0x55f66dc1f400 2024-09-13T23:45:16.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.775 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:45:16.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 20 zone default found 2024-09-13T23:45:16.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 4 Realm: () 2024-09-13T23:45:16.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:45:16.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:45:16.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:45:16.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 20 zonegroup default 2024-09-13T23:45:16.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.776 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb1400 con 0x55f66dbb0400 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb1800 con 0x55f66dbb0000 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb1c00 con 0x55f66dbb0000 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 20 started sync module instance, tier type = 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:45:16.778 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.770+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb1800 con 0x55f66dc1f400 2024-09-13T23:45:16.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.774+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.774+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94516890893056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb1c00 con 0x55f66dc1f400 2024-09-13T23:45:16.781 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.774+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb0c00 con 0x55f66dc1f000 2024-09-13T23:45:16.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.778+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.778+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94516890893056] v35'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.778+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94516890894208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb1000 con 0x55f66dc1f000 2024-09-13T23:45:16.784 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.778+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb1800 con 0x55f66dc1f400 2024-09-13T23:45:16.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.778+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.778+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94516890895360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1fc00 con 0x55f66dc1f400 2024-09-13T23:45:16.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.778+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94516890894208] v35'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.787 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.778+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dbb0400 2024-09-13T23:45:16.789 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.782+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94516890895360] v35'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.782+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.782+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94516890896512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb0800 con 0x55f66dbb0400 2024-09-13T23:45:16.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.782+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb1400 con 0x55f66dc1f400 2024-09-13T23:45:16.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.786+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94516890896512] v35'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.786+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.786+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94516890897664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb0c00 con 0x55f66dc1f400 2024-09-13T23:45:16.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.786+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb1000 con 0x55f66dbb0000 2024-09-13T23:45:16.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.786+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.786+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94516890898816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dce6000 con 0x55f66dbb0000 2024-09-13T23:45:16.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.786+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dce6400 con 0x55f66dbb0000 2024-09-13T23:45:16.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.786+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94516890897664] v35'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.790+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94516890898816] v35'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.790+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.790+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94516890899968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dce6800 con 0x55f66dbb0000 2024-09-13T23:45:16.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.790+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dce6c00 con 0x55f66dbb0400 2024-09-13T23:45:16.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.790+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'231 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94516890901120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dce7000 con 0x55f66dbb0400 2024-09-13T23:45:16.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cfcfaf40 20 add_watcher() i=0 2024-09-13T23:45:16.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cfcfaf40 20 add_watcher() i=1 2024-09-13T23:45:16.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cfcfaf40 20 add_watcher() i=2 2024-09-13T23:45:16.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cfcfaf40 20 add_watcher() i=3 2024-09-13T23:45:16.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cfcfaf40 20 add_watcher() i=4 2024-09-13T23:45:16.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cfcfaf40 20 add_watcher() i=5 2024-09-13T23:45:16.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94516890899968] v35'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cfcfaf40 20 add_watcher() i=6 2024-09-13T23:45:16.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94516890901120] v35'232 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cfcfaf40 20 add_watcher() i=7 2024-09-13T23:45:16.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cfcfaf40 2 all 8 watchers are set, enabling cache 2024-09-13T23:45:16.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dce7400 con 0x55f66dc1f000 2024-09-13T23:45:16.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.794+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94516892581888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dce7800 con 0x55f66dc1f000 2024-09-13T23:45:16.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94516892581888] v35'230 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56b5481700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:45:16.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56cfcfaf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:45:16.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56cfcfaf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:45:16.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56cfcfaf40 1 Processor -- start 2024-09-13T23:45:16.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56cfcfaf40 1 -- start start 2024-09-13T23:45:16.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56cfcfaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 0x55f66db5b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56cfcfaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d898780 con 0x55f66dce6c00 2024-09-13T23:45:16.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56cdcb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 0x55f66db5b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56cdcb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 0x55f66db5b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43044/0 (socket says 172.21.15.167:43044) 2024-09-13T23:45:16.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56cdcb2700 1 -- 172.21.15.167:0/4036957444 learned_addr learned my addr 172.21.15.167:0/4036957444 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56cdcb2700 1 -- 172.21.15.167:0/4036957444 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f66d884b60 con 0x55f66dce6c00 2024-09-13T23:45:16.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56cdcb2700 1 --2- 172.21.15.167:0/4036957444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 0x55f66db5b180 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55f66d9ce0c0 tx=0x55f66d977860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846be721fe798934 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56b4c80700 1 -- 172.21.15.167:0/4036957444 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66d917760 con 0x55f66dce6c00 2024-09-13T23:45:16.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56b4c80700 1 -- 172.21.15.167:0/4036957444 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f66dc20600 con 0x55f66dce6c00 2024-09-13T23:45:16.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.798+0000 7f56b4c80700 1 -- 172.21.15.167:0/4036957444 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66dcfc000 con 0x55f66dce6c00 2024-09-13T23:45:16.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/4036957444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 msgr2=0x55f66db5b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/4036957444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 0x55f66db5b180 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x55f66d9ce0c0 tx=0x55f66d977860 comp rx=0 tx=0).stop 2024-09-13T23:45:16.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/4036957444 shutdown_connections 2024-09-13T23:45:16.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/4036957444 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 0x55f66db5b180 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/4036957444 >> 172.21.15.167:0/4036957444 conn(0x55f66dce7c00 msgr2=0x55f66d974000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/4036957444 shutdown_connections 2024-09-13T23:45:16.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/4036957444 wait complete. 2024-09-13T23:45:16.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 Processor -- start 2024-09-13T23:45:16.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 -- start start 2024-09-13T23:45:16.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 0x55f66db5b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d900000 con 0x55f66dce6c00 2024-09-13T23:45:16.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cdcb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 0x55f66db5b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cdcb2700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 0x55f66db5b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43058/0 (socket says 172.21.15.167:43058) 2024-09-13T23:45:16.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cdcb2700 1 -- 172.21.15.167:0/841964170 learned_addr learned my addr 172.21.15.167:0/841964170 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cdcb2700 1 -- 172.21.15.167:0/841964170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f66d884d00 con 0x55f66dce6c00 2024-09-13T23:45:16.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cdcb2700 1 --2- 172.21.15.167:0/841964170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 0x55f66db5b180 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x55f66dcfe150 tx=0x55f66dcc2420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56b347d700 1 -- 172.21.15.167:0/841964170 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66dcfc160 con 0x55f66dce6c00 2024-09-13T23:45:16.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56b347d700 1 -- 172.21.15.167:0/841964170 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f66dc20300 con 0x55f66dce6c00 2024-09-13T23:45:16.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/841964170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f66d884340 con 0x55f66dce6c00 2024-09-13T23:45:16.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56b347d700 1 -- 172.21.15.167:0/841964170 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f66dcfc2c0 con 0x55f66dce6c00 2024-09-13T23:45:16.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.802+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/841964170 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f66d885d40 con 0x55f66dce6c00 2024-09-13T23:45:16.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.806+0000 7f56b347d700 1 -- 172.21.15.167:0/841964170 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f66d96db80 con 0x55f66dce6c00 2024-09-13T23:45:16.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.806+0000 7f56b347d700 1 --2- 172.21.15.167:0/841964170 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dd72000 0x55f66dd78000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.806+0000 7f56b347d700 1 -- 172.21.15.167:0/841964170 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f66d8d9500 con 0x55f66dce6c00 2024-09-13T23:45:16.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.806+0000 7f56cecb4700 1 --2- 172.21.15.167:0/841964170 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dd72000 0x55f66dd78000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.806+0000 7f56cfcfaf40 5 note: GC not initialized 2024-09-13T23:45:16.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.806+0000 7f56cecb4700 1 --2- 172.21.15.167:0/841964170 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dd72000 0x55f66dd78000 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55f66dc83c50 tx=0x55f66dbd37a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.806+0000 7f56b0c78700 20 reqs_thread_entry: start 2024-09-13T23:45:16.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.806+0000 7f56cfcfaf40 20 init_complete bucket index max shards: 11 2024-09-13T23:45:16.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.806+0000 7f56cfcfaf40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:45:16.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.806+0000 7f56aec74700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:45:47 2024 2024-09-13T23:45:16.815 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:45:16.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.806+0000 7f56ae452700 20 reqs_thread_entry: start 2024-09-13T23:45:16.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.810+0000 7f56cfcfaf40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.810+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dce6000 con 0x55f66dc1f000 2024-09-13T23:45:16.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.810+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.810+0000 7f56cfcfaf40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:45:16.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.810+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dce6400 con 0x55f66dbb0000 2024-09-13T23:45:16.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.810+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(63 usage.10 [call out=15b] v0'0 uv32 ondisk = 0) v8 ==== 152+0+15 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.810+0000 7f56b5481700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:45:16.818 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.810+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94516892581888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb0800 con 0x55f66dc1f000 2024-09-13T23:45:16.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.814+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94516892581888] v35'231 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.814+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94516890893056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dbb1000 con 0x55f66dc1f400 2024-09-13T23:45:16.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.818+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94516890893056] v35'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.818+0000 7f56cfcfaf40 20 remove_watcher() i=0 2024-09-13T23:45:16.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.818+0000 7f56cfcfaf40 2 removed watcher, disabling cache 2024-09-13T23:45:16.824 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.818+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94516890894208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dc1f800 con 0x55f66dc1f000 2024-09-13T23:45:16.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.818+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94516890894208] v35'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dc1f000 2024-09-13T23:45:16.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.818+0000 7f56cfcfaf40 20 remove_watcher() i=1 2024-09-13T23:45:16.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.818+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94516890895360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dd72400 con 0x55f66dc1f400 2024-09-13T23:45:16.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.822+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94516890895360] v35'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.822+0000 7f56cfcfaf40 20 remove_watcher() i=2 2024-09-13T23:45:16.828 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.822+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94516890896512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dd72800 con 0x55f66dbb0400 2024-09-13T23:45:16.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.822+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94516890896512] v35'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.822+0000 7f56cfcfaf40 20 remove_watcher() i=3 2024-09-13T23:45:16.831 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.822+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94516890897664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dd72c00 con 0x55f66dc1f400 2024-09-13T23:45:16.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.826+0000 7f56ce4b3700 1 -- 172.21.15.167:0/1740297274 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94516890897664] v35'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd200 con 0x55f66dc1f400 2024-09-13T23:45:16.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.826+0000 7f56cfcfaf40 20 remove_watcher() i=4 2024-09-13T23:45:16.833 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.826+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94516890898816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dd73000 con 0x55f66dbb0000 2024-09-13T23:45:16.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.830+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94516890898816] v35'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.830+0000 7f56cfcfaf40 20 remove_watcher() i=5 2024-09-13T23:45:16.836 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.830+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94516890899968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dd73400 con 0x55f66dbb0000 2024-09-13T23:45:16.838 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.830+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94516890899968] v35'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dccd440 con 0x55f66dbb0000 2024-09-13T23:45:16.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.830+0000 7f56cfcfaf40 20 remove_watcher() i=6 2024-09-13T23:45:16.839 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.830+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94516890901120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f66dd73800 con 0x55f66dbb0400 2024-09-13T23:45:16.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cecb4700 1 -- 172.21.15.167:0/1740297274 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94516890901120] v35'233 uv231 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55f66dcccb40 con 0x55f66dbb0400 2024-09-13T23:45:16.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 20 remove_watcher() i=7 2024-09-13T23:45:16.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f66dc1f400 msgr2=0x55f66db5a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.842 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f66dc1f400 0x55f66db5a100 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f66dc1f000 msgr2=0x55f66db59b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55f66dc1f000 0x55f66db59b80 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f66dbb0400 msgr2=0x55f66db5ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55f66dbb0400 0x55f66db5ac00 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f66dbb0000 msgr2=0x55f66db5a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55f66dbb0000 0x55f66db5a680 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dc1ec00 msgr2=0x55f66db59600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dc1ec00 0x55f66db59600 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x55f66db53920 tx=0x55f66d977800 comp rx=0 tx=0).stop 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 msgr2=0x55f66db58b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 0x55f66db58b00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x55f66e6ea420 tx=0x55f66dbd2840 comp rx=0 tx=0).stop 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cdcb2700 1 -- 172.21.15.167:0/1740297274 reap_dead start 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 shutdown_connections 2024-09-13T23:45:16.843 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/1740297274 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873800 0x55f66db58b00 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 >> 172.21.15.167:0/1740297274 conn(0x55f66d873400 msgr2=0x55f66d973800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 shutdown_connections 2024-09-13T23:45:16.844 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/1740297274 wait complete. 2024-09-13T23:45:16.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/841964170 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dd72000 msgr2=0x55f66dd78000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.834+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/841964170 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dd72000 0x55f66dd78000 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55f66dc83c50 tx=0x55f66dbd37a0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.845 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/841964170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 msgr2=0x55f66db5b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/841964170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 0x55f66db5b180 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x55f66dcfe150 tx=0x55f66dcc2420 comp rx=0 tx=0).stop 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/841964170 shutdown_connections 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/841964170 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dd72000 0x55f66dd78000 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/841964170 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66dce6c00 0x55f66db5b180 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/841964170 >> 172.21.15.167:0/841964170 conn(0x55f66dce7c00 msgr2=0x55f66d974000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/841964170 shutdown_connections 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/841964170 wait complete. 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/2880621218 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dc1e000 msgr2=0x55f66db59080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/2880621218 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dc1e000 0x55f66db59080 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55f66e6eb5f0 tx=0x55f66dbd2240 comp rx=0 tx=0).stop 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/2880621218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 msgr2=0x55f66db58580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/2880621218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 0x55f66db58580 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x55f66d9cfec0 tx=0x55f66d977a40 comp rx=0 tx=0).stop 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/2880621218 shutdown_connections 2024-09-13T23:45:16.846 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/2880621218 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f66dc1e000 0x55f66db59080 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 --2- 172.21.15.167:0/2880621218 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f66d873000 0x55f66db58580 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/2880621218 >> 172.21.15.167:0/2880621218 conn(0x55f66d872c00 msgr2=0x55f66d973000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/2880621218 shutdown_connections 2024-09-13T23:45:16.847 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.838+0000 7f56cfcfaf40 1 -- 172.21.15.167:0/2880621218 wait complete. 2024-09-13T23:45:16.847 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:45:16.847 INFO:teuthology.orchestra.run.smithi167.stdout: "entries": [], 2024-09-13T23:45:16.847 INFO:teuthology.orchestra.run.smithi167.stdout: "summary": [] 2024-09-13T23:45:16.848 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:45:16.850 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-09-13T23:45:16.850 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo', '--purge-data'] 2024-09-13T23:45:16.850 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo', '--purge-data'] 2024-09-13T23:45:16.850 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo --purge-data 2024-09-13T23:45:16.932 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:45:16.932 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:45:16.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.926+0000 7f2ee5ad0f40 1 Processor -- start 2024-09-13T23:45:16.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.926+0000 7f2ee5ad0f40 1 -- start start 2024-09-13T23:45:16.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.926+0000 7f2ee5ad0f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5633e431d000 0x5633e4602580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.926+0000 7f2ee5ad0f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x5633e43e1500 con 0x5633e431d400 2024-09-13T23:45:16.933 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.926+0000 7f2ee5ad0f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5633e43aaa00 con 0x5633e431d000 2024-09-13T23:45:16.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.926+0000 7f2ee4289700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x5633e431d000 0x5633e4602580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.926+0000 7f2ee3a88700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x5633e431d400 0x5633e441c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:38768/0 (socket says 172.21.15.167:38768) 2024-09-13T23:45:16.934 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.926+0000 7f2ee3a88700 1 -- 172.21.15.167:0/3055807876 learned_addr learned my addr 172.21.15.167:0/3055807876 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:45:16.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.926+0000 7f2ee4289700 1 -- 172.21.15.167:0/3055807876 >> v1:172.21.15.12:6789/0 conn(0x5633e431d400 legacy=0x5633e441c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.926+0000 7f2ee4289700 1 -- 172.21.15.167:0/3055807876 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e432fd40 con 0x5633e431d000 2024-09-13T23:45:16.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.926+0000 7f2ee4289700 1 --2- 172.21.15.167:0/3055807876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x5633e45fda70 tx=0x5633e4421920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fbcba3cffdbd14f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee3287700 1 -- 172.21.15.167:0/3055807876 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e43c14a0 con 0x5633e431d000 2024-09-13T23:45:16.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee3287700 1 -- 172.21.15.167:0/3055807876 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633e43e1800 con 0x5633e431d000 2024-09-13T23:45:16.935 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee3287700 1 -- 172.21.15.167:0/3055807876 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e43c1600 con 0x5633e431d000 2024-09-13T23:45:16.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/3055807876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 msgr2=0x5633e4602580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/3055807876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x5633e45fda70 tx=0x5633e4421920 comp rx=0 tx=0).stop 2024-09-13T23:45:16.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/3055807876 shutdown_connections 2024-09-13T23:45:16.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/3055807876 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.936 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/3055807876 >> 172.21.15.167:0/3055807876 conn(0x5633e431cc00 msgr2=0x5633e441c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/3055807876 shutdown_connections 2024-09-13T23:45:16.937 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/3055807876 wait complete. 2024-09-13T23:45:16.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee5ad0f40 1 Processor -- start 2024-09-13T23:45:16.941 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee5ad0f40 1 -- start start 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee5ad0f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee5ad0f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e43aa500 con 0x5633e431d000 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee4289700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee4289700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43068/0 (socket says 172.21.15.167:43068) 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.930+0000 7f2ee4289700 1 -- 172.21.15.167:0/332960732 learned_addr learned my addr 172.21.15.167:0/332960732 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee4289700 1 -- 172.21.15.167:0/332960732 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e432fba0 con 0x5633e431d000 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee4289700 1 --2- 172.21.15.167:0/332960732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x5633e517d800 tx=0x5633e4421800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3572fef5cbb64f3c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee2a86700 1 -- 172.21.15.167:0/332960732 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e43c1760 con 0x5633e431d000 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee2a86700 1 -- 172.21.15.167:0/332960732 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633e43e1b00 con 0x5633e431d000 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee2a86700 1 -- 172.21.15.167:0/332960732 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e43c14a0 con 0x5633e431d000 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/332960732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 msgr2=0x5633e4602580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/332960732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x5633e517d800 tx=0x5633e4421800 comp rx=0 tx=0).stop 2024-09-13T23:45:16.942 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/332960732 shutdown_connections 2024-09-13T23:45:16.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/332960732 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/332960732 >> 172.21.15.167:0/332960732 conn(0x5633e431cc00 msgr2=0x5633e441d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/332960732 shutdown_connections 2024-09-13T23:45:16.943 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/332960732 wait complete. 2024-09-13T23:45:16.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee5ad0f40 1 Processor -- start 2024-09-13T23:45:16.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee5ad0f40 1 -- start start 2024-09-13T23:45:16.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee5ad0f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.934+0000 7f2ee5ad0f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e43aa3c0 con 0x5633e431d000 2024-09-13T23:45:16.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee4289700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee4289700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43080/0 (socket says 172.21.15.167:43080) 2024-09-13T23:45:16.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee4289700 1 -- 172.21.15.167:0/4111009861 learned_addr learned my addr 172.21.15.167:0/4111009861 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.944 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee4289700 1 -- 172.21.15.167:0/4111009861 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e432fa00 con 0x5633e431d000 2024-09-13T23:45:16.945 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee4289700 1 --2- 172.21.15.167:0/4111009861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x5633e517d6b0 tx=0x5633e4421920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee1283700 1 -- 172.21.15.167:0/4111009861 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e43c1600 con 0x5633e431d000 2024-09-13T23:45:16.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee1283700 1 -- 172.21.15.167:0/4111009861 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633e43e1c80 con 0x5633e431d000 2024-09-13T23:45:16.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4111009861 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5633e432f860 con 0x5633e431d000 2024-09-13T23:45:16.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee1283700 1 -- 172.21.15.167:0/4111009861 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e43c18c0 con 0x5633e431d000 2024-09-13T23:45:16.946 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4111009861 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633e432f6c0 con 0x5633e431d000 2024-09-13T23:45:16.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee5ad0f40 1 Processor -- start 2024-09-13T23:45:16.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee5ad0f40 1 -- start start 2024-09-13T23:45:16.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee5ad0f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 0x5633e4602b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.947 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.938+0000 7f2ee5ad0f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e4342640 con 0x5633e431d800 2024-09-13T23:45:16.948 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee1283700 1 -- 172.21.15.167:0/4111009861 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5633e4416b00 con 0x5633e431d000 2024-09-13T23:45:16.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee4a8a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 0x5633e4602b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee4a8a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 0x5633e4602b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43092/0 (socket says 172.21.15.167:43092) 2024-09-13T23:45:16.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/646495228 learned_addr learned my addr 172.21.15.167:0/646495228 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee1283700 1 --2- 172.21.15.167:0/4111009861 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e46c2000 0x5633e4603080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee1283700 1 -- 172.21.15.167:0/4111009861 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5633e4383180 con 0x5633e431d000 2024-09-13T23:45:16.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee4289700 1 --2- 172.21.15.167:0/4111009861 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e46c2000 0x5633e4603080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/646495228 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e432e000 con 0x5633e431d800 2024-09-13T23:45:16.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee4a8a700 1 --2- 172.21.15.167:0/646495228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 0x5633e4602b00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x5633e5193bf0 tx=0x5633e4684b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f95415889956e4d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee4289700 1 --2- 172.21.15.167:0/4111009861 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e46c2000 0x5633e4603080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x5633e5193110 tx=0x5633e4684c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2edea7e700 1 -- 172.21.15.167:0/646495228 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e43c1600 con 0x5633e431d800 2024-09-13T23:45:16.949 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2edea7e700 1 -- 172.21.15.167:0/646495228 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633e46dc000 con 0x5633e431d800 2024-09-13T23:45:16.950 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2edea7e700 1 -- 172.21.15.167:0/646495228 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e43c18c0 con 0x5633e431d800 2024-09-13T23:45:16.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/646495228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 msgr2=0x5633e4602b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:16.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/646495228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 0x5633e4602b00 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x5633e5193bf0 tx=0x5633e4684b40 comp rx=0 tx=0).stop 2024-09-13T23:45:16.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/646495228 shutdown_connections 2024-09-13T23:45:16.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/646495228 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 0x5633e4602b00 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:16.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/646495228 >> 172.21.15.167:0/646495228 conn(0x5633e431d400 msgr2=0x5633e441d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:16.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/646495228 shutdown_connections 2024-09-13T23:45:16.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/646495228 wait complete. 2024-09-13T23:45:16.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee5ad0f40 1 Processor -- start 2024-09-13T23:45:16.951 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.942+0000 7f2ee5ad0f40 1 -- start start 2024-09-13T23:45:16.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.946+0000 7f2ee5ad0f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 0x5633e4602b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.946+0000 7f2ee5ad0f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e43428c0 con 0x5633e431d800 2024-09-13T23:45:16.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.946+0000 7f2ee4a8a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 0x5633e4602b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.952 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.946+0000 7f2ee4a8a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 0x5633e4602b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43102/0 (socket says 172.21.15.167:43102) 2024-09-13T23:45:16.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.946+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 learned_addr learned my addr 172.21.15.167:0/1061798046 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:16.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.946+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e432eea0 con 0x5633e431d800 2024-09-13T23:45:16.953 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.946+0000 7f2ee4a8a700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 0x5633e4602b00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x5633e51926c0 tx=0x5633e46845a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.946+0000 7f2edd27b700 1 -- 172.21.15.167:0/1061798046 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e43c1760 con 0x5633e431d800 2024-09-13T23:45:16.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.946+0000 7f2edd27b700 1 -- 172.21.15.167:0/1061798046 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633e46dc180 con 0x5633e431d800 2024-09-13T23:45:16.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.946+0000 7f2edd27b700 1 -- 172.21.15.167:0/1061798046 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e43c1600 con 0x5633e431d800 2024-09-13T23:45:16.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.946+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5633e432e820 con 0x5633e431d800 2024-09-13T23:45:16.954 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.946+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633e432e9c0 con 0x5633e431d800 2024-09-13T23:45:16.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2edd27b700 1 -- 172.21.15.167:0/1061798046 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5633e4417080 con 0x5633e431d800 2024-09-13T23:45:16.956 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2edd27b700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e46c2c00 0x5633e4603600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2edd27b700 1 -- 172.21.15.167:0/1061798046 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5633e4383340 con 0x5633e431d800 2024-09-13T23:45:16.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee3a88700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e46c2c00 0x5633e4603600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5633e46c3000 0x5633e4603b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3400 con 0x5633e46c3000 2024-09-13T23:45:16.957 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee4a8a700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5633e46c3000 0x5633e4603b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee3a88700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e46c2c00 0x5633e4603600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5633e46bfe30 tx=0x5633e47720c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee4a8a700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5633e46c3000 0x5633e4603b80 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.958 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 20 realm 2024-09-13T23:45:16.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3000 2024-09-13T23:45:16.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:45:16.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3c00 con 0x5633e46c3000 2024-09-13T23:45:16.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.959 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5633e46c3400 0x5633e4604100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3400 2024-09-13T23:45:16.960 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.950+0000 7f2ee4289700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5633e46c3400 0x5633e4604100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee4289700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5633e46c3400 0x5633e4604100 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee5ad0f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:45:16.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.961 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3c00 con 0x5633e46c3400 2024-09-13T23:45:16.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee5ad0f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:45:16.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee5ad0f40 20 searching for the correct realm 2024-09-13T23:45:16.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5633e4788000 0x5633e4604680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788400 con 0x5633e4788000 2024-09-13T23:45:16.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee3a88700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5633e4788000 0x5633e4604680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.962 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee3a88700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5633e4788000 0x5633e4604680 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:16.963 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.954+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788400 con 0x5633e46c3400 2024-09-13T23:45:16.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4289700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5633e4788400 0x5633e4604c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:16.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788400 2024-09-13T23:45:16.964 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4a8a700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5633e4788400 0x5633e4604c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:16.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4a8a700 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5633e4788400 0x5633e4604c00 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:16.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:16.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3000 2024-09-13T23:45:16.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.965 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3000 2024-09-13T23:45:16.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788000 2024-09-13T23:45:16.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:16.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3000 2024-09-13T23:45:16.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.966 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.958+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3400 2024-09-13T23:45:16.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.962+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.967 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.962+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3400 2024-09-13T23:45:16.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.962+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.962+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3000 2024-09-13T23:45:16.968 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.962+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.962+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788400 2024-09-13T23:45:16.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.962+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:16.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.962+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3000 2024-09-13T23:45:16.969 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.962+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.962+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788400 2024-09-13T23:45:16.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.962+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:16.970 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.962+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3400 2024-09-13T23:45:16.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.971 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788000 2024-09-13T23:45:16.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:16.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3000 2024-09-13T23:45:16.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.972 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3400 2024-09-13T23:45:16.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788000 2024-09-13T23:45:16.973 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3000 2024-09-13T23:45:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.974 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788400 2024-09-13T23:45:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.966+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3000 2024-09-13T23:45:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.975 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788000 2024-09-13T23:45:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788400 2024-09-13T23:45:16.976 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3000 2024-09-13T23:45:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788400 2024-09-13T23:45:16.977 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:16.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788000 2024-09-13T23:45:16.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:16.978 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788400 2024-09-13T23:45:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.970+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3400 2024-09-13T23:45:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.979 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3400 2024-09-13T23:45:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.980 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788000 2024-09-13T23:45:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:16.981 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3400 2024-09-13T23:45:16.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.982 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e46c3400 2024-09-13T23:45:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee5ad0f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:45:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee5ad0f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:45:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee5ad0f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:45:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee5ad0f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:45:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.983 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.974+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788800 con 0x5633e46c3000 2024-09-13T23:45:16.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788c00 con 0x5633e46c3400 2024-09-13T23:45:16.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.984 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:45:16.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789000 con 0x5633e46c3400 2024-09-13T23:45:16.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:45:16.985 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 20 zone default found 2024-09-13T23:45:16.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 4 Realm: () 2024-09-13T23:45:16.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:45:16.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:45:16.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:45:16.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 20 zonegroup default 2024-09-13T23:45:16.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.986 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789400 con 0x5633e4788400 2024-09-13T23:45:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789800 con 0x5633e4788000 2024-09-13T23:45:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:16.987 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.978+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789c00 con 0x5633e4788000 2024-09-13T23:45:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.982+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.982+0000 7f2ee5ad0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.982+0000 7f2ee5ad0f40 20 started sync module instance, tier type = 2024-09-13T23:45:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.982+0000 7f2ee5ad0f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:45:16.988 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.982+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789800 con 0x5633e46c3400 2024-09-13T23:45:16.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.982+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.982+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94780874791680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789c00 con 0x5633e46c3400 2024-09-13T23:45:16.991 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.982+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788c00 con 0x5633e46c3000 2024-09-13T23:45:16.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.986+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94780874791680] v35'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.986+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.986+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94780874792832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789000 con 0x5633e46c3000 2024-09-13T23:45:16.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.986+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789800 con 0x5633e46c3400 2024-09-13T23:45:16.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.990+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94780874792832] v35'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:16.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.990+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:16.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.990+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94780874793984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3c00 con 0x5633e46c3400 2024-09-13T23:45:16.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.990+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3800 con 0x5633e4788400 2024-09-13T23:45:16.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.994+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:16.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.994+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94780874795136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788800 con 0x5633e4788400 2024-09-13T23:45:16.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.994+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789400 con 0x5633e46c3400 2024-09-13T23:45:17.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.994+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94780874793984] v35'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.994+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94780874795136] v35'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.994+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.994+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94780874796288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788c00 con 0x5633e46c3400 2024-09-13T23:45:17.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.994+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789000 con 0x5633e4788000 2024-09-13T23:45:17.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.998+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94780874796288] v35'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.998+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.998+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94780874797440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478e000 con 0x5633e4788000 2024-09-13T23:45:17.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.998+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478e400 con 0x5633e4788000 2024-09-13T23:45:17.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:16.998+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94780874797440] v35'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94780874798592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478e800 con 0x5633e4788000 2024-09-13T23:45:17.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478ec00 con 0x5633e4788400 2024-09-13T23:45:17.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'234 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94780874798592] v35'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94780874799744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f000 con 0x5633e4788400 2024-09-13T23:45:17.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 20 add_watcher() i=0 2024-09-13T23:45:17.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 20 add_watcher() i=1 2024-09-13T23:45:17.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 20 add_watcher() i=2 2024-09-13T23:45:17.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 20 add_watcher() i=3 2024-09-13T23:45:17.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 20 add_watcher() i=4 2024-09-13T23:45:17.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 20 add_watcher() i=5 2024-09-13T23:45:17.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 20 add_watcher() i=6 2024-09-13T23:45:17.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94780874799744] v35'235 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 20 add_watcher() i=7 2024-09-13T23:45:17.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:45:17.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.002+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f400 con 0x5633e46c3000 2024-09-13T23:45:17.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.006+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:17.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.006+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94780876472320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f800 con 0x5633e46c3000 2024-09-13T23:45:17.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.006+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94780876472320] v35'232 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:17.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.006+0000 7f2ecb257700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:45:17.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.006+0000 7f2ee5ad0f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:45:17.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.006+0000 7f2ee5ad0f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:45:17.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.006+0000 7f2ee5ad0f40 1 Processor -- start 2024-09-13T23:45:17.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.006+0000 7f2ee5ad0f40 1 -- start start 2024-09-13T23:45:17.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.006+0000 7f2ee5ad0f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 0x5633e4605180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee5ad0f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e43aa3c0 con 0x5633e478ec00 2024-09-13T23:45:17.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee3a88700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 0x5633e4605180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee3a88700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 0x5633e4605180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43116/0 (socket says 172.21.15.167:43116) 2024-09-13T23:45:17.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee3a88700 1 -- 172.21.15.167:0/4076528244 learned_addr learned my addr 172.21.15.167:0/4076528244 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee3a88700 1 -- 172.21.15.167:0/4076528244 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e432eb60 con 0x5633e478ec00 2024-09-13T23:45:17.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee3a88700 1 --2- 172.21.15.167:0/4076528244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 0x5633e4605180 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x5633e479e090 tx=0x5633e4772060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3562496739f9e87a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ecaa56700 1 -- 172.21.15.167:0/4076528244 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e47a6000 con 0x5633e478ec00 2024-09-13T23:45:17.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ecaa56700 1 -- 172.21.15.167:0/4076528244 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633e46dc600 con 0x5633e478ec00 2024-09-13T23:45:17.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ecaa56700 1 -- 172.21.15.167:0/4076528244 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e47a6160 con 0x5633e478ec00 2024-09-13T23:45:17.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4076528244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 msgr2=0x5633e4605180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/4076528244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 0x5633e4605180 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x5633e479e090 tx=0x5633e4772060 comp rx=0 tx=0).stop 2024-09-13T23:45:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4076528244 shutdown_connections 2024-09-13T23:45:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/4076528244 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 0x5633e4605180 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4076528244 >> 172.21.15.167:0/4076528244 conn(0x5633e478fc00 msgr2=0x5633e441e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4076528244 shutdown_connections 2024-09-13T23:45:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4076528244 wait complete. 2024-09-13T23:45:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee5ad0f40 1 Processor -- start 2024-09-13T23:45:17.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee5ad0f40 1 -- start start 2024-09-13T23:45:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.010+0000 7f2ee5ad0f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 0x5633e4605180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ee5ad0f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633e43aa500 con 0x5633e478ec00 2024-09-13T23:45:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ee3a88700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 0x5633e4605180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ee3a88700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 0x5633e4605180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43130/0 (socket says 172.21.15.167:43130) 2024-09-13T23:45:17.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ee3a88700 1 -- 172.21.15.167:0/2100146805 learned_addr learned my addr 172.21.15.167:0/2100146805 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ee3a88700 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633e432ed00 con 0x5633e478ec00 2024-09-13T23:45:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ee3a88700 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 0x5633e4605180 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x5633e479f350 tx=0x5633e4772180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ec9253700 1 -- 172.21.15.167:0/2100146805 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e47a62c0 con 0x5633e478ec00 2024-09-13T23:45:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ec9253700 1 -- 172.21.15.167:0/2100146805 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633e46dc300 con 0x5633e478ec00 2024-09-13T23:45:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ec9253700 1 -- 172.21.15.167:0/2100146805 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5633e47a6000 con 0x5633e478ec00 2024-09-13T23:45:17.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5633e432e340 con 0x5633e478ec00 2024-09-13T23:45:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633e432eb60 con 0x5633e478ec00 2024-09-13T23:45:17.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ec9253700 1 -- 172.21.15.167:0/2100146805 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x5633e4417b80 con 0x5633e478ec00 2024-09-13T23:45:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ec9253700 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e4820000 0x5633e4826000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ec9253700 1 -- 172.21.15.167:0/2100146805 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5633e4383500 con 0x5633e478ec00 2024-09-13T23:45:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.014+0000 7f2ee4a8a700 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e4820000 0x5633e4826000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.018+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5633e478f000 0x5633e4605700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.018+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f400 con 0x5633e478f000 2024-09-13T23:45:17.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.018+0000 7f2ee4289700 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5633e478f000 0x5633e4605700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.018+0000 7f2ee4a8a700 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e4820000 0x5633e4826000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5633e5192f60 tx=0x5633e4685500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.018+0000 7f2ee4289700 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5633e478f000 0x5633e4605700 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.018+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 gc.0 [create,call,call,call] v35'40 uv33 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.018+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5633e478e000 0x5633e4826580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.018+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:2 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478e400 con 0x5633e478e000 2024-09-13T23:45:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.018+0000 7f2ee3a88700 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5633e478e000 0x5633e4826580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.022+0000 7f2ee3a88700 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5633e478e000 0x5633e4826580 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.022+0000 7f2ee3a88700 1 -- 172.21.15.167:0/2100146805 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(2 gc.1 [create,call,call,call] v35'29 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e478e000 2024-09-13T23:45:17.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.022+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5633e4788800 0x5633e4826b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.022+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:3 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789000 con 0x5633e4788800 2024-09-13T23:45:17.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.022+0000 7f2ee4a8a700 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5633e4788800 0x5633e4826b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.022+0000 7f2ee4a8a700 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5633e4788800 0x5633e4826b00 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.026+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/2100146805 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(3 gc.2 [create,call,call,call] v35'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788800 2024-09-13T23:45:17.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.026+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5633e46c3800 0x5633e4827080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.026+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820400 con 0x5633e46c3800 2024-09-13T23:45:17.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.026+0000 7f2ee4289700 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5633e46c3800 0x5633e4827080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.026+0000 7f2ee4289700 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5633e46c3800 0x5633e4827080 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.030+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(4 gc.3 [create,call,call,call] v35'233 uv166 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3800 2024-09-13T23:45:17.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.030+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820800 con 0x5633e478f000 2024-09-13T23:45:17.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.030+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 gc.4 [create,call,call,call] v35'74 uv55 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.030+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820c00 con 0x5633e4788800 2024-09-13T23:45:17.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.034+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/2100146805 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(6 gc.5 [create,call,call,call] v35'52 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788800 2024-09-13T23:45:17.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.034+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821000 con 0x5633e478f000 2024-09-13T23:45:17.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.038+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 gc.6 [create,call,call,call] v35'30 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.043 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.038+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821400 con 0x5633e46c3800 2024-09-13T23:45:17.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.038+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(8 gc.7 [create,call,call,call] v35'50 uv35 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3800 2024-09-13T23:45:17.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.038+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:9 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821800 con 0x5633e478f000 2024-09-13T23:45:17.048 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.042+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(9 gc.8 [create,call,call,call] v35'28 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.042+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:10 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821c00 con 0x5633e478f000 2024-09-13T23:45:17.051 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.046+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(10 gc.9 [create,call,call,call] v35'35 uv28 ondisk = -125 ((125) Operation canceled)) v8 ==== 274+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.046+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:11 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f400 con 0x5633e478f000 2024-09-13T23:45:17.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.046+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(11 gc.10 [create,call,call,call] v35'75 uv51 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.046+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:12 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789c00 con 0x5633e4788800 2024-09-13T23:45:17.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.050+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/2100146805 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(12 gc.11 [create,call,call,call] v35'34 uv24 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788800 2024-09-13T23:45:17.056 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.050+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789800 con 0x5633e478f000 2024-09-13T23:45:17.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.054+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(13 gc.12 [create,call,call,call] v35'31 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.059 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.054+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789400 con 0x5633e46c3800 2024-09-13T23:45:17.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.054+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(14 gc.13 [create,call,call,call] v35'56 uv40 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3800 2024-09-13T23:45:17.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.054+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788c00 con 0x5633e46c3800 2024-09-13T23:45:17.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.058+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(15 gc.14 [create,call,call,call] v35'234 uv162 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3800 2024-09-13T23:45:17.066 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.058+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:16 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3c00 con 0x5633e478f000 2024-09-13T23:45:17.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.062+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(16 gc.15 [create,call,call,call] v35'71 uv45 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.062+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:17 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f400 con 0x5633e478f000 2024-09-13T23:45:17.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.062+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(17 gc.16 [create,call,call,call] v35'76 uv53 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.062+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:18 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820400 con 0x5633e4788800 2024-09-13T23:45:17.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.066+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/2100146805 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(18 gc.17 [create,call,call,call] v35'30 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788800 2024-09-13T23:45:17.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.066+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820800 con 0x5633e478f000 2024-09-13T23:45:17.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.070+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(19 gc.18 [create,call,call,call] v35'37 uv29 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.070+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821000 con 0x5633e46c3800 2024-09-13T23:45:17.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.070+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(20 gc.19 [create,call,call,call] v35'235 uv164 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3800 2024-09-13T23:45:17.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.070+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821400 con 0x5633e478f000 2024-09-13T23:45:17.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.074+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(21 gc.20 [create,call,call,call] v35'72 uv47 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.074+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821800 con 0x5633e478f000 2024-09-13T23:45:17.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.074+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(22 gc.21 [create,call,call,call] v35'38 uv31 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.074+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821c00 con 0x5633e4788800 2024-09-13T23:45:17.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.078+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/2100146805 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(23 gc.22 [create,call,call,call] v35'30 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788800 2024-09-13T23:45:17.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.078+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:24 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820c00 con 0x5633e478f000 2024-09-13T23:45:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.078+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(24 gc.23 [create,call,call,call] v35'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.078+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:25 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820400 con 0x5633e4788800 2024-09-13T23:45:17.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.082+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/2100146805 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(25 gc.24 [create,call,call,call] v35'24 uv17 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788800 2024-09-13T23:45:17.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.082+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821c00 con 0x5633e4788800 2024-09-13T23:45:17.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.082+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/2100146805 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(26 gc.25 [create,call,call,call] v35'53 uv39 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788800 2024-09-13T23:45:17.090 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.082+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820800 con 0x5633e4788800 2024-09-13T23:45:17.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.086+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/2100146805 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(27 gc.26 [create,call,call,call] v35'30 uv23 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788800 2024-09-13T23:45:17.092 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.086+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:28 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821000 con 0x5633e478f000 2024-09-13T23:45:17.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.090+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(28 gc.27 [create,call,call,call] v35'29 uv21 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.095 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.090+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821400 con 0x5633e46c3800 2024-09-13T23:45:17.097 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.090+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(29 gc.28 [create,call,call,call] v35'51 uv37 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3800 2024-09-13T23:45:17.098 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.090+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:30 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f800 con 0x5633e478f000 2024-09-13T23:45:17.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.094+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(30 gc.29 [create,call,call,call] v35'73 uv49 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e478f000 2024-09-13T23:45:17.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.094+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:31 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f400 con 0x5633e46c3800 2024-09-13T23:45:17.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.094+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(31 gc.30 [create,call,call,call] v35'57 uv42 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3800 2024-09-13T23:45:17.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.094+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3c00 con 0x5633e478e000 2024-09-13T23:45:17.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.098+0000 7f2ee3a88700 1 -- 172.21.15.167:0/2100146805 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(32 gc.31 [create,call,call,call] v35'34 uv27 ondisk = -125 ((125) Operation canceled)) v8 ==== 275+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e478e000 2024-09-13T23:45:17.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ec6a4e700 20 reqs_thread_entry: start 2024-09-13T23:45:17.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 20 init_complete bucket index max shards: 11 2024-09-13T23:45:17.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:45:17.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ec4a4a700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:45:47 2024 2024-09-13T23:45:17.108 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:45:17.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ec4228700 20 reqs_thread_entry: start 2024-09-13T23:45:17.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-09-13T23:45:17.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.108 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789800 con 0x5633e46c3000 2024-09-13T23:45:17.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call out=48b,read 0~357 out=357b,getxattrs out=75b] v0'0 uv15 ondisk = 0) v8 ==== 231+0+480 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:17.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 20 rados_obj.operate() r=0 bl.length=357 2024-09-13T23:45:17.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-09-13T23:45:17.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-09-13T23:45:17.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:45:17.109 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789400 con 0x5633e4788400 2024-09-13T23:45:17.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv41 ondisk = 0) v8 ==== 155+0+183 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 : miss 2024-09-13T23:45:17.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 8.14 8:28ad8365:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788c00 con 0x5633e46c3400 2024-09-13T23:45:17.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 [call out=48b,stat out=16b,read 0~352 out=352b,getxattrs out=218b] v0'0 uv4 ondisk = 0) v8 ==== 332+0+634 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 20 rados_obj.operate() r=0 bl.length=352 2024-09-13T23:45:17.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 info.flags=0x17 2024-09-13T23:45:17.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 10 adding default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 to cache LRU end 2024-09-13T23:45:17.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:45:17.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:45:17.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.102+0000 7f2ee5ad0f40 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 2024-09-13T23:45:17.111 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.106+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 4.3d 4:bf64a068:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4952000 con 0x5633e46c3400 2024-09-13T23:45:17.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.106+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.112 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.106+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:66 4.1c 4:3a54506f:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4952400 con 0x5633e4788400 2024-09-13T23:45:17.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.106+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(66 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.106+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 4.35 4:ad9bb58e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4952800 con 0x5633e46c3000 2024-09-13T23:45:17.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.106+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(67 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:17.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.106+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:68 4.2e 4:7597960b:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4952c00 con 0x5633e4788000 2024-09-13T23:45:17.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.106+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(68 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.106+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:69 4.2d 4:b679e55e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4953000 con 0x5633e4788400 2024-09-13T23:45:17.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.110+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 15 ==== osd_op_reply(69 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4 [call out=426b] v0'0 uv4 ondisk = 0) v8 ==== 194+0+426 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.110+0000 7f2ee5ad0f40 10 cls_bucket_list_unordered: got 15f6f8dc036519d7fe15b39338f6e5db 2024-09-13T23:45:17.115 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.110+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:70 4.d 4:b2eff8c8:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4953400 con 0x5633e46c3400 2024-09-13T23:45:17.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.110+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(70 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.110+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 4.33 4:cf23bc19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4953800 con 0x5633e4788400 2024-09-13T23:45:17.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.110+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 16 ==== osd_op_reply(71 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.6 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.110+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:72 4.2d 4:b4d5e757:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4953c00 con 0x5633e4788400 2024-09-13T23:45:17.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.110+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 17 ==== osd_op_reply(72 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.119 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.110+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 4.a 4:5066eb5a:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4952000 con 0x5633e46c3400 2024-09-13T23:45:17.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(73 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.8 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:74 4.11 4:89425d89:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4953400 con 0x5633e4788000 2024-09-13T23:45:17.120 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(74 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.9 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:75 4.13 4:c8da7602:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4952400 con 0x5633e4788400 2024-09-13T23:45:17.121 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 18 ==== osd_op_reply(75 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee5ad0f40 20 get_obj_state: rctx=0x5633e43ad730 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x5633e48d61e8 s->prefetch_data=0 2024-09-13T23:45:17.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 3.27s0 3:e5ec1c9f:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6_15f6f8dc036519d7fe15b39338f6e5db:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4952800 con 0x5633e4788000 2024-09-13T23:45:17.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 16 ==== osd_op_reply(76 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6_15f6f8dc036519d7fe15b39338f6e5db [getxattrs out=879b,stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 262+0+895 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.122 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee5ad0f40 10 manifest: total_size = 6 2024-09-13T23:45:17.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee5ad0f40 20 get_obj_state: setting s->obj_tag to 9144208c-3c85-4fb6-9007-2693d5a930f0.4222.16170881361610805990 2024-09-13T23:45:17.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee5ad0f40 20 get_obj_state: rctx=0x5633e43ad730 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x5633e48d61e8 s->prefetch_data=0 2024-09-13T23:45:17.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee5ad0f40 20 prepare_atomic_modification: state is not atomic. state=0x5633e48d61e8 2024-09-13T23:45:17.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee5ad0f40 20 bucket index object: default.rgw.buckets.index:.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4 2024-09-13T23:45:17.123 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.114+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:77 4.2d 4:b679e55e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788c00 con 0x5633e4788400 2024-09-13T23:45:17.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.118+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 19 ==== osd_op_reply(77 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4 [stat,call,call] v35'5 uv5 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.126 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.118+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:78 3.27s0 3:e5ec1c9f:::9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6_15f6f8dc036519d7fe15b39338f6e5db:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789800 con 0x5633e4788000 2024-09-13T23:45:17.128 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 17 ==== osd_op_reply(78 9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6_15f6f8dc036519d7fe15b39338f6e5db [call] v35'7 uv7 ondisk = 0) v8 ==== 220+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:79 4.2d 4:b679e55e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821800 con 0x5633e4788400 2024-09-13T23:45:17.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee5ad0f40 20 list_objects_ordered: starting attempt 1 2024-09-13T23:45:17.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee5ad0f40 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-09-13T23:45:17.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:80 4.3d 4:bf64a068:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820c00 con 0x5633e46c3400 2024-09-13T23:45:17.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:81 4.1c 4:3a54506f:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821000 con 0x5633e4788400 2024-09-13T23:45:17.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:82 4.35 4:ad9bb58e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821400 con 0x5633e46c3000 2024-09-13T23:45:17.129 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:83 4.2e 4:7597960b:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820400 con 0x5633e4788000 2024-09-13T23:45:17.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:84 4.2d 4:b679e55e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821c00 con 0x5633e4788400 2024-09-13T23:45:17.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:85 4.d 4:b2eff8c8:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820800 con 0x5633e46c3400 2024-09-13T23:45:17.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== osd_op_reply(80 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(82 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:17.130 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 18 ==== osd_op_reply(83 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:86 4.33 4:cf23bc19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f800 con 0x5633e4788400 2024-09-13T23:45:17.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 20 ==== osd_op_reply(81 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.122+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(85 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.131 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:87 4.2d 4:b4d5e757:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f400 con 0x5633e4788400 2024-09-13T23:45:17.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:88 4.a 4:5066eb5a:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4953000 con 0x5633e46c3400 2024-09-13T23:45:17.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:89 4.11 4:89425d89:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4953800 con 0x5633e4788000 2024-09-13T23:45:17.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:90 4.13 4:c8da7602:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4953c00 con 0x5633e4788400 2024-09-13T23:45:17.132 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 21 ==== osd_op_reply(86 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.6 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 22 ==== osd_op_reply(87 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 19 ==== osd_op_reply(89 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.9 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.133 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 23 ==== osd_op_reply(79 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4 [stat,call,call] v35'6 uv6 ondisk = 0) v8 ==== 278+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 24 ==== osd_op_reply(90 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(88 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.8 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ede27d700 20 handle_completion(): completion ok for obj=15f6f8dc036519d7fe15b39338f6e5db 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 25 ==== osd_op_reply(84 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4 [call out=124b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6])"; uploads.size()=0; is_truncated=0 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 10 lifecycle: RGWRados::convert_old_bucket_info(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6]) 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 10 lifecycle: cache get: name=default.rgw.meta+root+myfoo : miss 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 20 lifecycle: rados->read ofs=0 len=0 2024-09-13T23:45:17.134 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:91 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4952400 con 0x5633e4788400 2024-09-13T23:45:17.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 26 ==== osd_op_reply(91 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv39 ondisk = 0) v8 ==== 275+0+304 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 20 lifecycle: rados_obj.operate() r=0 bl.length=165 2024-09-13T23:45:17.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 10 lifecycle: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-09-13T23:45:17.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 10 lifecycle: adding default.rgw.meta+root+myfoo to cache LRU end 2024-09-13T23:45:17.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 10 lifecycle: updating xattr: name=ceph.objclass.version bl.length()=42 2024-09-13T23:45:17.135 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.126+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:92 8.14 8:28ad8365:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~352 in=352b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4953400 con 0x5633e46c3400 2024-09-13T23:45:17.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== osd_op_reply(92 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 [delete,create,call,call,writefull 0~352,setxattr (123)] v35'8 uv8 ondisk = 0) v8 ==== 416+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee5ad0f40 10 lifecycle: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 info.flags=0x17 2024-09-13T23:45:17.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee5ad0f40 10 lifecycle: moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 to cache LRU end 2024-09-13T23:45:17.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee5ad0f40 10 lifecycle: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:45:17.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee5ad0f40 10 lifecycle: distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6, ofs0, ns] 2024-09-13T23:45:17.137 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:93 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94780876475776 in=737b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478e400 con 0x5633e46c3400 2024-09-13T23:45:17.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== watch-notify(notify (1) cookie 94780874791680 notify 150323855403 ret 0) v3 ==== 767+0+0 (crc 0 0 0) 0x5633e46791e0 con 0x5633e46c3400 2024-09-13T23:45:17.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ede27d700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855403 cookie 94780874791680 notifier 4854 bl.length()=725 2024-09-13T23:45:17.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(93 notify.0 [notify cookie 94780876475776 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ede27d700 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 info.flags=0x17 2024-09-13T23:45:17.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ede27d700 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 to cache LRU end 2024-09-13T23:45:17.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ede27d700 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-09-13T23:45:17.138 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ede27d700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:94 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789400 con 0x5633e46c3400 2024-09-13T23:45:17.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(94 notify.0 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 31 ==== watch-notify(notify_complete (2) cookie 94780876475776 notify 150323855403 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5633e4679380 con 0x5633e46c3400 2024-09-13T23:45:17.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee5ad0f40 20 lifecycle: handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6]): orig_sources= new_sources= 2024-09-13T23:45:17.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee5ad0f40 20 lifecycle: handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6]): potential sources added=[] removed=[] 2024-09-13T23:45:17.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee5ad0f40 20 lifecycle: handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6]): orig_dests= new_dests= 2024-09-13T23:45:17.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee5ad0f40 20 lifecycle: handle_bi_update(): bucket=:myfoo[9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6]): potential dests added=[] removed=[] 2024-09-13T23:45:17.139 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.130+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:33 6.16 6:6d4431ed:lc::lc.7:head [call lock.lock in=65b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821800 con 0x5633e4788800 2024-09-13T23:45:17.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.134+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/2100146805 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(33 lc.7 [call] v35'31 uv31 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788800 2024-09-13T23:45:17.144 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.134+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:34 6.16 6:6d4431ed:lc::lc.7:head [call rgw.lc_rm_entry in=92b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821000 con 0x5633e4788800 2024-09-13T23:45:17.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/2100146805 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(34 lc.7 [call] v35'32 uv32 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788800 2024-09-13T23:45:17.145 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 6.16 6:6d4431ed:lc::lc.7:head [call lock.unlock in=49b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821400 con 0x5633e4788800 2024-09-13T23:45:17.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/2100146805 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(35 lc.7 [call] v35'33 uv33 ondisk = 0) v8 ==== 148+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788800 2024-09-13T23:45:17.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:95 4.3d 4:bf64a068:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788c00 con 0x5633e46c3400 2024-09-13T23:45:17.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:96 4.1c 4:3a54506f:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4864000 con 0x5633e4788400 2024-09-13T23:45:17.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:97 4.35 4:ad9bb58e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4864400 con 0x5633e46c3000 2024-09-13T23:45:17.147 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:98 4.2e 4:7597960b:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4864800 con 0x5633e4788000 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:99 4.2d 4:b679e55e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4864c00 con 0x5633e4788400 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:100 4.d 4:b2eff8c8:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4865000 con 0x5633e46c3400 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:101 4.33 4:cf23bc19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4865400 con 0x5633e4788400 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:102 4.2d 4:b4d5e757:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4865800 con 0x5633e4788400 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:103 4.a 4:5066eb5a:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4865c00 con 0x5633e46c3400 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:104 4.11 4:89425d89:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789800 con 0x5633e4788000 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:105 4.13 4:c8da7602:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789400 con 0x5633e4788400 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 20 ==== osd_op_reply(98 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.3 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 32 ==== osd_op_reply(95 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.0 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 21 ==== osd_op_reply(104 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.9 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 27 ==== osd_op_reply(96 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.1 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 28 ==== osd_op_reply(99 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4 [call out=124b] v0'0 uv6 ondisk = 0) v8 ==== 194+0+124 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.148 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.138+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 33 ==== osd_op_reply(100 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.5 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.142+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 34 ==== osd_op_reply(103 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.8 [call out=85b] v0'0 uv3 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.142+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 29 ==== osd_op_reply(101 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.6 [call out=85b] v0'0 uv5 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.142+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 30 ==== osd_op_reply(105 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.10 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.142+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 31 ==== osd_op_reply(102 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.7 [call out=85b] v0'0 uv2 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.142+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(97 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.2 [call out=85b] v0'0 uv1 ondisk = 0) v8 ==== 194+0+85 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:17.149 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.142+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:106 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788c00 con 0x5633e4788400 2024-09-13T23:45:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.142+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 32 ==== osd_op_reply(106 foo.buckets [call] v35'44 uv44 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.142+0000 7f2ee5ad0f40 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x17) 2024-09-13T23:45:17.150 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.142+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:107 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f800 con 0x5633e4788400 2024-09-13T23:45:17.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.146+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 33 ==== osd_op_reply(107 myfoo [call,call,delete] v35'45 uv45 ondisk = 0) v8 ==== 233+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.146+0000 7f2ee5ad0f40 10 removing default.rgw.meta+root+myfoo from cache 2024-09-13T23:45:17.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.146+0000 7f2ee5ad0f40 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-09-13T23:45:17.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.146+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:108 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94780876475776 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f400 con 0x5633e4788000 2024-09-13T23:45:17.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.146+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 22 ==== watch-notify(notify (1) cookie 94780874798592 notify 150323855384 ret 0) v3 ==== 191+0+0 (crc 0 0 0) 0x5633e47ecb60 con 0x5633e4788000 2024-09-13T23:45:17.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.146+0000 7f2ede27d700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855384 cookie 94780874798592 notifier 4854 bl.length()=149 2024-09-13T23:45:17.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.146+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 23 ==== osd_op_reply(108 notify.6 [notify cookie 94780876475776 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.152 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.146+0000 7f2ede27d700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:109 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48f6000 con 0x5633e4788000 2024-09-13T23:45:17.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.146+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 24 ==== watch-notify(notify_complete (2) cookie 94780876475776 notify 150323855384 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5633e47ecd00 con 0x5633e4788000 2024-09-13T23:45:17.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.146+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 25 ==== osd_op_reply(109 notify.6 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.153 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.146+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:110 8.14 8:28ad8365:root::.bucket.meta.myfoo%3a9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478e400 con 0x5633e46c3400 2024-09-13T23:45:17.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 35 ==== osd_op_reply(110 .bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 [call,call,delete] v35'9 uv9 ondisk = 0) v8 ==== 290+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 10 removing default.rgw.meta+root+.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 from cache 2024-09-13T23:45:17.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6, ofs0, ns] 2024-09-13T23:45:17.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:111 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94780876475776 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e46c3c00 con 0x5633e46c3400 2024-09-13T23:45:17.156 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 36 ==== watch-notify(notify (1) cookie 94780874791680 notify 150323855404 ret 0) v3 ==== 248+0+0 (crc 0 0 0) 0x5633e4679380 con 0x5633e46c3400 2024-09-13T23:45:17.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ede27d700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855404 cookie 94780874791680 notifier 4854 bl.length()=206 2024-09-13T23:45:17.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 37 ==== osd_op_reply(111 notify.0 [notify cookie 94780876475776 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ede27d700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:112 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48f6400 con 0x5633e46c3400 2024-09-13T23:45:17.157 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 38 ==== osd_op_reply(112 notify.0 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 39 ==== watch-notify(notify_complete (2) cookie 94780876475776 notify 150323855404 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5633e4679520 con 0x5633e46c3400 2024-09-13T23:45:17.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:113 4.3d 4:bf64a068:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820400 con 0x5633e46c3400 2024-09-13T23:45:17.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:114 4.1c 4:3a54506f:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821c00 con 0x5633e4788400 2024-09-13T23:45:17.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:115 4.35 4:ad9bb58e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821800 con 0x5633e46c3000 2024-09-13T23:45:17.158 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:116 4.2e 4:7597960b:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821000 con 0x5633e4788000 2024-09-13T23:45:17.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:117 4.2d 4:b679e55e:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821400 con 0x5633e4788400 2024-09-13T23:45:17.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:118 4.d 4:b2eff8c8:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820c00 con 0x5633e46c3400 2024-09-13T23:45:17.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:119 4.33 4:cf23bc19:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4820800 con 0x5633e4788400 2024-09-13T23:45:17.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:120 4.2d 4:b4d5e757:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4864800 con 0x5633e4788400 2024-09-13T23:45:17.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:121 4.a 4:5066eb5a:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4864000 con 0x5633e46c3400 2024-09-13T23:45:17.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:122 4.11 4:89425d89:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48f6c00 con 0x5633e4788000 2024-09-13T23:45:17.159 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.150+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:123 4.13 4:c8da7602:::.dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48f7000 con 0x5633e4788400 2024-09-13T23:45:17.160 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.154+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(115 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.2 [delete] v35'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:17.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.154+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 40 ==== osd_op_reply(121 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.8 [delete] v35'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.154+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 26 ==== osd_op_reply(122 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.9 [delete] v35'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.154+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 34 ==== osd_op_reply(123 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.10 [delete] v35'2 uv2 ondisk = 0) v8 ==== 195+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.161 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.154+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 41 ==== osd_op_reply(118 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.5 [delete] v35'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.154+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 35 ==== osd_op_reply(114 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.1 [delete] v35'2 uv2 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.154+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 36 ==== osd_op_reply(119 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.6 [delete] v35'6 uv6 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.163 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.154+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 42 ==== osd_op_reply(113 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.0 [delete] v35'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 27 ==== osd_op_reply(116 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.3 [delete] v35'4 uv4 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 37 ==== osd_op_reply(117 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.4 [delete] v35'7 uv7 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 38 ==== osd_op_reply(120 .dir.9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6.7 [delete] v35'8 uv8 ondisk = 0) v8 ==== 194+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee5ad0f40 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 : miss 2024-09-13T23:45:17.164 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee5ad0f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:124 6.3 6:c6e91263:::pubsub..bucket.myfoo%2f9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48f7400 con 0x5633e46c3400 2024-09-13T23:45:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 43 ==== osd_op_reply(124 pubsub..bucket.myfoo/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee5ad0f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee5ad0f40 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 info.flags=0x0 2024-09-13T23:45:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee5ad0f40 10 adding default.rgw.log++pubsub..bucket.myfoo/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 to cache LRU end 2024-09-13T23:45:17.165 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:125 6.3 6:c6e91263:::pubsub..bucket.myfoo%2f9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48f7800 con 0x5633e46c3400 2024-09-13T23:45:17.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 44 ==== osd_op_reply(125 pubsub..bucket.myfoo/9144208c-3c85-4fb6-9007-2693d5a930f0.4224.6 [delete] v35'77 uv55 ondisk = -2 ((2) No such file or directory)) v8 ==== 208+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.166 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.158+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:126 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=58b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48f7c00 con 0x5633e4788400 2024-09-13T23:45:17.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.162+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 39 ==== osd_op_reply(126 foo.buckets [call] v35'46 uv46 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.162+0000 7f2ee5ad0f40 10 removing key index: 9te6NH5mcdcq0Tc5i8i1 2024-09-13T23:45:17.168 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.162+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:127 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821c00 con 0x5633e46c3000 2024-09-13T23:45:17.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.162+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(127 9te6NH5mcdcq0Tc5i8i1 [delete] v35'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:17.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.162+0000 7f2ee5ad0f40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-09-13T23:45:17.170 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.162+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:128 7.e 7:7759931f:::notify.3:head [notify cookie 94780876475776 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821400 con 0x5633e4788400 2024-09-13T23:45:17.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.162+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 40 ==== watch-notify(notify (1) cookie 94780874795136 notify 150323855396 ret 0) v3 ==== 212+0+0 (crc 0 0 0) 0x5633e4725380 con 0x5633e4788400 2024-09-13T23:45:17.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.162+0000 7f2ede27d700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855396 cookie 94780874795136 notifier 4854 bl.length()=170 2024-09-13T23:45:17.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.162+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 41 ==== osd_op_reply(128 notify.3 [notify cookie 94780876475776 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.162+0000 7f2ede27d700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:129 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48f6800 con 0x5633e4788400 2024-09-13T23:45:17.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.162+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 42 ==== osd_op_reply(129 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.171 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 43 ==== watch-notify(notify_complete (2) cookie 94780876475776 notify 150323855396 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5633e4725520 con 0x5633e4788400 2024-09-13T23:45:17.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2ee5ad0f40 10 removing email index: foo@foo.com 2024-09-13T23:45:17.172 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:130 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4821800 con 0x5633e46c3000 2024-09-13T23:45:17.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 24 ==== osd_op_reply(130 foo@foo.com [delete] v35'2 uv2 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:17.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2ee5ad0f40 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-09-13T23:45:17.173 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:131 7.e 7:7759931f:::notify.3:head [notify cookie 94780876475776 in=174b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e478f800 con 0x5633e4788400 2024-09-13T23:45:17.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 44 ==== watch-notify(notify (1) cookie 94780874795136 notify 150323855397 ret 0) v3 ==== 204+0+0 (crc 0 0 0) 0x5633e4725520 con 0x5633e4788400 2024-09-13T23:45:17.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 45 ==== osd_op_reply(131 notify.3 [notify cookie 94780876475776 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2edea7e700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855397 cookie 94780874795136 notifier 4854 bl.length()=162 2024-09-13T23:45:17.174 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2edea7e700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:132 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48fe000 con 0x5633e4788400 2024-09-13T23:45:17.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 46 ==== watch-notify(notify_complete (2) cookie 94780876475776 notify 150323855397 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5633e47256c0 con 0x5633e4788400 2024-09-13T23:45:17.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2ee5ad0f40 10 removing user buckets index 2024-09-13T23:45:17.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 47 ==== osd_op_reply(132 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.175 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.166+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:133 8.9 8:96d98480:users.uid::foo.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789400 con 0x5633e4788400 2024-09-13T23:45:17.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.170+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 48 ==== osd_op_reply(133 foo.buckets [delete] v35'47 uv47 ondisk = 0) v8 ==== 155+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.170+0000 7f2ee5ad0f40 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:foo.buckets, ofs0, ns] 2024-09-13T23:45:17.176 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.170+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:134 7.3 7:c609908c:::notify.5:head [notify cookie 94780876475776 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788c00 con 0x5633e4788000 2024-09-13T23:45:17.177 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.170+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 28 ==== watch-notify(notify (1) cookie 94780874797440 notify 150323855385 ret 0) v3 ==== 202+0+0 (crc 0 0 0) 0x5633e47ecd00 con 0x5633e4788000 2024-09-13T23:45:17.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.170+0000 7f2edea7e700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855385 cookie 94780874797440 notifier 4854 bl.length()=160 2024-09-13T23:45:17.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.170+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 29 ==== osd_op_reply(134 notify.5 [notify cookie 94780876475776 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.170+0000 7f2edea7e700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:135 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48fe400 con 0x5633e4788000 2024-09-13T23:45:17.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.170+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 30 ==== osd_op_reply(135 notify.5 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.170+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 31 ==== watch-notify(notify_complete (2) cookie 94780876475776 notify 150323855385 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5633e47ecea0 con 0x5633e4788000 2024-09-13T23:45:17.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.170+0000 7f2ee5ad0f40 10 removing user index: foo 2024-09-13T23:45:17.178 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.170+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:136 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4864400 con 0x5633e46c3000 2024-09-13T23:45:17.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.174+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 25 ==== osd_op_reply(136 foo [call,call,delete] v35'20 uv20 ondisk = 0) v8 ==== 231+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:17.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.174+0000 7f2ee5ad0f40 10 removing default.rgw.meta+users.uid+foo from cache 2024-09-13T23:45:17.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.174+0000 7f2ee5ad0f40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-09-13T23:45:17.179 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.174+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:137 7.d 7:b4812045:::notify.4:head [notify cookie 94780876475776 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4864c00 con 0x5633e46c3400 2024-09-13T23:45:17.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.174+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 45 ==== watch-notify(notify (1) cookie 94780874796288 notify 150323855405 ret 0) v3 ==== 194+0+0 (crc 0 0 0) 0x5633e4679520 con 0x5633e46c3400 2024-09-13T23:45:17.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.174+0000 7f2edea7e700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855405 cookie 94780874796288 notifier 4854 bl.length()=152 2024-09-13T23:45:17.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.174+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 46 ==== osd_op_reply(137 notify.4 [notify cookie 94780876475776 out=8b] v0'0 uv234 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.180 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.174+0000 7f2edea7e700 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:138 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4865000 con 0x5633e46c3400 2024-09-13T23:45:17.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.174+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 47 ==== watch-notify(notify_complete (2) cookie 94780876475776 notify 150323855405 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x5633e46796c0 con 0x5633e46c3400 2024-09-13T23:45:17.181 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.174+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 48 ==== osd_op_reply(138 notify.4 [notify-ack] v0'0 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.174+0000 7f2ecb257700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:45:17.183 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.178+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:139 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94780876472320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4865800 con 0x5633e46c3000 2024-09-13T23:45:17.185 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.178+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 26 ==== osd_op_reply(139 data_loggenerations_metadata [watch unwatch cookie 94780876472320] v35'236 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:17.186 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.178+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:140 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94780874791680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48fe000 con 0x5633e46c3400 2024-09-13T23:45:17.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.182+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 49 ==== osd_op_reply(140 notify.0 [watch unwatch cookie 94780874791680] v35'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.182+0000 7f2ee5ad0f40 20 remove_watcher() i=0 2024-09-13T23:45:17.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.182+0000 7f2ee5ad0f40 2 removed watcher, disabling cache 2024-09-13T23:45:17.188 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.182+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:141 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94780874792832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48fe800 con 0x5633e46c3000 2024-09-13T23:45:17.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.182+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.1 v2:172.21.15.167:6800/3875661733 27 ==== osd_op_reply(141 notify.1 [watch unwatch cookie 94780874792832] v35'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e46c3000 2024-09-13T23:45:17.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.182+0000 7f2ee5ad0f40 20 remove_watcher() i=1 2024-09-13T23:45:17.190 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.182+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:142 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94780874793984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48fec00 con 0x5633e46c3400 2024-09-13T23:45:17.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.186+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 50 ==== osd_op_reply(142 notify.2 [watch unwatch cookie 94780874793984] v35'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.186+0000 7f2ee5ad0f40 20 remove_watcher() i=2 2024-09-13T23:45:17.193 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.186+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:143 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94780874795136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48ff000 con 0x5633e4788400 2024-09-13T23:45:17.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.190+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 49 ==== osd_op_reply(143 notify.3 [watch unwatch cookie 94780874795136] v35'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.195 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.190+0000 7f2ee5ad0f40 20 remove_watcher() i=3 2024-09-13T23:45:17.196 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.190+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:144 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94780874796288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e48ff400 con 0x5633e46c3400 2024-09-13T23:45:17.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.190+0000 7f2ee4289700 1 -- 172.21.15.167:0/1061798046 <== osd.0 v2:172.21.15.12:6800/2649399468 51 ==== osd_op_reply(144 notify.4 [watch unwatch cookie 94780874796288] v35'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775200 con 0x5633e46c3400 2024-09-13T23:45:17.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.190+0000 7f2ee5ad0f40 20 remove_watcher() i=4 2024-09-13T23:45:17.198 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.190+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:145 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94780874797440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789800 con 0x5633e4788000 2024-09-13T23:45:17.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.194+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 32 ==== osd_op_reply(145 notify.5 [watch unwatch cookie 94780874797440] v35'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.200 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.194+0000 7f2ee5ad0f40 20 remove_watcher() i=5 2024-09-13T23:45:17.201 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.194+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:146 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94780874798592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4788c00 con 0x5633e4788000 2024-09-13T23:45:17.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.194+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 <== osd.3 v2:172.21.15.167:6816/3688363524 33 ==== osd_op_reply(146 notify.6 [watch unwatch cookie 94780874798592] v35'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4775440 con 0x5633e4788000 2024-09-13T23:45:17.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.194+0000 7f2ee5ad0f40 20 remove_watcher() i=6 2024-09-13T23:45:17.203 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:147 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94780874799744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5633e4789400 con 0x5633e4788400 2024-09-13T23:45:17.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee4a8a700 1 -- 172.21.15.167:0/1061798046 <== osd.2 v2:172.21.15.167:6808/1180369139 50 ==== osd_op_reply(147 notify.7 [watch unwatch cookie 94780874799744] v35'236 uv234 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x5633e4774b40 con 0x5633e4788400 2024-09-13T23:45:17.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 20 remove_watcher() i=7 2024-09-13T23:45:17.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5633e46c3400 msgr2=0x5633e4604100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5633e46c3400 0x5633e4604100 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5633e46c3000 msgr2=0x5633e4603b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.206 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5633e46c3000 0x5633e4603b80 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5633e4788400 msgr2=0x5633e4604c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5633e4788400 0x5633e4604c00 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5633e4788000 msgr2=0x5633e4604680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5633e4788000 0x5633e4604680 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e46c2c00 msgr2=0x5633e4603600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e46c2c00 0x5633e4603600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5633e46bfe30 tx=0x5633e47720c0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 msgr2=0x5633e4602b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 0x5633e4602b00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x5633e51926c0 tx=0x5633e46845a0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee3a88700 1 -- 172.21.15.167:0/1061798046 reap_dead start 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 shutdown_connections 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/1061798046 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d800 0x5633e4602b00 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 >> 172.21.15.167:0/1061798046 conn(0x5633e431d400 msgr2=0x5633e441d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 shutdown_connections 2024-09-13T23:45:17.207 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/1061798046 wait complete. 2024-09-13T23:45:17.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5633e478f000 msgr2=0x5633e4605700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5633e478f000 0x5633e4605700 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5633e46c3800 msgr2=0x5633e4827080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.209 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x5633e46c3800 0x5633e4827080 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5633e478e000 msgr2=0x5633e4826580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x5633e478e000 0x5633e4826580 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5633e4788800 msgr2=0x5633e4826b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.198+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x5633e4788800 0x5633e4826b00 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e4820000 msgr2=0x5633e4826000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e4820000 0x5633e4826000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x5633e5192f60 tx=0x5633e4685500 comp rx=0 tx=0).stop 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 msgr2=0x5633e4605180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee4289700 1 -- 172.21.15.167:0/2100146805 reap_dead start 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 0x5633e4605180 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x5633e479f350 tx=0x5633e4772180 comp rx=0 tx=0).stop 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 shutdown_connections 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/2100146805 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e478ec00 0x5633e4605180 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 >> 172.21.15.167:0/2100146805 conn(0x5633e478fc00 msgr2=0x5633e441e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.210 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 shutdown_connections 2024-09-13T23:45:17.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/2100146805 wait complete. 2024-09-13T23:45:17.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4111009861 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e46c2000 msgr2=0x5633e4603080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/4111009861 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e46c2000 0x5633e4603080 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x5633e5193110 tx=0x5633e4684c60 comp rx=0 tx=0).stop 2024-09-13T23:45:17.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4111009861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 msgr2=0x5633e4602580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/4111009861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x5633e517d6b0 tx=0x5633e4421920 comp rx=0 tx=0).stop 2024-09-13T23:45:17.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4111009861 shutdown_connections 2024-09-13T23:45:17.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/4111009861 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5633e46c2000 0x5633e4603080 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 --2- 172.21.15.167:0/4111009861 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x5633e431d000 0x5633e4602580 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4111009861 >> 172.21.15.167:0/4111009861 conn(0x5633e431cc00 msgr2=0x5633e441d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4111009861 shutdown_connections 2024-09-13T23:45:17.211 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.202+0000 7f2ee5ad0f40 1 -- 172.21.15.167:0/4111009861 wait complete. 2024-09-13T23:45:17.214 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-09-13T23:45:17.214 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-09-13T23:45:17.214 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-09-13T23:45:17.255 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:45:17.255 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:45:17.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3c1792f40 1 Processor -- start 2024-09-13T23:45:17.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3c1792f40 1 -- start start 2024-09-13T23:45:17.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3c1792f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x564f302ed400 0x564f305d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3c1792f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x564f303b1500 con 0x564f302ed000 2024-09-13T23:45:17.256 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3c1792f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x564f3037aa00 con 0x564f302ed400 2024-09-13T23:45:17.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3bf74a700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x564f302ed400 0x564f305d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3bff4b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x564f302ed000 0x564f303ec800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:38776/0 (socket says 172.21.15.167:38776) 2024-09-13T23:45:17.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3bff4b700 1 -- 172.21.15.167:0/10465628 learned_addr learned my addr 172.21.15.167:0/10465628 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:45:17.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3bef49700 1 -- 172.21.15.167:0/10465628 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1659700054 0 0) 0x564f30448000 con 0x564f302ed000 2024-09-13T23:45:17.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3bef49700 1 -- 172.21.15.167:0/10465628 --> v1:172.21.15.12:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x564f303b1680 con 0x564f302ed000 2024-09-13T23:45:17.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3bf74a700 1 -- 172.21.15.167:0/10465628 >> v1:172.21.15.12:6789/0 conn(0x564f302ed000 legacy=0x564f303ec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3bf74a700 1 -- 172.21.15.167:0/10465628 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f302ffd40 con 0x564f302ed400 2024-09-13T23:45:17.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3bf74a700 1 --2- 172.21.15.167:0/10465628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed400 0x564f305d2580 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x564f305cd920 tx=0x564f303f1980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e543277ba55640 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3bef49700 1 -- 172.21.15.167:0/10465628 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f303914a0 con 0x564f302ed400 2024-09-13T23:45:17.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3bef49700 1 -- 172.21.15.167:0/10465628 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f303b1500 con 0x564f302ed400 2024-09-13T23:45:17.258 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3bef49700 1 -- 172.21.15.167:0/10465628 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f30391600 con 0x564f302ed400 2024-09-13T23:45:17.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3c1792f40 1 -- 172.21.15.167:0/10465628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed400 msgr2=0x564f305d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/10465628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed400 0x564f305d2580 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x564f305cd920 tx=0x564f303f1980 comp rx=0 tx=0).stop 2024-09-13T23:45:17.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3c1792f40 1 -- 172.21.15.167:0/10465628 shutdown_connections 2024-09-13T23:45:17.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/10465628 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed400 0x564f305d2580 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3c1792f40 1 -- 172.21.15.167:0/10465628 >> 172.21.15.167:0/10465628 conn(0x564f302ecc00 msgr2=0x564f303ec000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3c1792f40 1 -- 172.21.15.167:0/10465628 shutdown_connections 2024-09-13T23:45:17.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.250+0000 7fb3c1792f40 1 -- 172.21.15.167:0/10465628 wait complete. 2024-09-13T23:45:17.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 Processor -- start 2024-09-13T23:45:17.260 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 -- start start 2024-09-13T23:45:17.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 0x564f305d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3037a500 con 0x564f302ed000 2024-09-13T23:45:17.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3bff4b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 0x564f305d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3bff4b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 0x564f305d2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43158/0 (socket says 172.21.15.167:43158) 2024-09-13T23:45:17.261 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3bff4b700 1 -- 172.21.15.167:0/3518520654 learned_addr learned my addr 172.21.15.167:0/3518520654 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3bff4b700 1 -- 172.21.15.167:0/3518520654 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f302ffba0 con 0x564f302ed000 2024-09-13T23:45:17.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3bff4b700 1 --2- 172.21.15.167:0/3518520654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 0x564f305d2580 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x564f310b7d70 tx=0x564f303f1b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab465cf783ec42eb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3be748700 1 -- 172.21.15.167:0/3518520654 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f303914a0 con 0x564f302ed000 2024-09-13T23:45:17.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3be748700 1 -- 172.21.15.167:0/3518520654 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f303b1b00 con 0x564f302ed000 2024-09-13T23:45:17.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3be748700 1 -- 172.21.15.167:0/3518520654 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f30391600 con 0x564f302ed000 2024-09-13T23:45:17.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 -- 172.21.15.167:0/3518520654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 msgr2=0x564f305d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/3518520654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 0x564f305d2580 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x564f310b7d70 tx=0x564f303f1b60 comp rx=0 tx=0).stop 2024-09-13T23:45:17.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 -- 172.21.15.167:0/3518520654 shutdown_connections 2024-09-13T23:45:17.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/3518520654 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 0x564f305d2580 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.263 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 -- 172.21.15.167:0/3518520654 >> 172.21.15.167:0/3518520654 conn(0x564f302ecc00 msgr2=0x564f303ed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 -- 172.21.15.167:0/3518520654 shutdown_connections 2024-09-13T23:45:17.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 -- 172.21.15.167:0/3518520654 wait complete. 2024-09-13T23:45:17.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 Processor -- start 2024-09-13T23:45:17.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 -- start start 2024-09-13T23:45:17.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 0x564f305d2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3c1792f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3037a3c0 con 0x564f302ed000 2024-09-13T23:45:17.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3bff4b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 0x564f305d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3bff4b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 0x564f305d2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43170/0 (socket says 172.21.15.167:43170) 2024-09-13T23:45:17.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.254+0000 7fb3bff4b700 1 -- 172.21.15.167:0/699787343 learned_addr learned my addr 172.21.15.167:0/699787343 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.264 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bff4b700 1 -- 172.21.15.167:0/699787343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f302ffa00 con 0x564f302ed000 2024-09-13T23:45:17.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bff4b700 1 --2- 172.21.15.167:0/699787343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 0x564f305d2580 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x564f310b7e00 tx=0x564f303f1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bcf45700 1 -- 172.21.15.167:0/699787343 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f303918c0 con 0x564f302ed000 2024-09-13T23:45:17.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bcf45700 1 -- 172.21.15.167:0/699787343 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f303b1c80 con 0x564f302ed000 2024-09-13T23:45:17.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3c1792f40 1 -- 172.21.15.167:0/699787343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564f302ff860 con 0x564f302ed000 2024-09-13T23:45:17.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bcf45700 1 -- 172.21.15.167:0/699787343 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f303914a0 con 0x564f302ed000 2024-09-13T23:45:17.266 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3c1792f40 1 -- 172.21.15.167:0/699787343 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f302ff6c0 con 0x564f302ed000 2024-09-13T23:45:17.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bcf45700 1 -- 172.21.15.167:0/699787343 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x564f303e6b00 con 0x564f302ed000 2024-09-13T23:45:17.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bcf45700 1 --2- 172.21.15.167:0/699787343 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f30684000 0x564f305d3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bcf45700 1 -- 172.21.15.167:0/699787343 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x564f30353180 con 0x564f302ed000 2024-09-13T23:45:17.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3c1792f40 1 Processor -- start 2024-09-13T23:45:17.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bf74a700 1 --2- 172.21.15.167:0/699787343 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f30684000 0x564f305d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3c1792f40 1 -- start start 2024-09-13T23:45:17.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3c1792f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 0x564f305d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3c1792f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3037a000 con 0x564f302ed800 2024-09-13T23:45:17.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bf74a700 1 --2- 172.21.15.167:0/699787343 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f30684000 0x564f305d3080 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x564f305cd920 tx=0x564f303f1800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bff4b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 0x564f305d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.267 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bff4b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 0x564f305d2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43178/0 (socket says 172.21.15.167:43178) 2024-09-13T23:45:17.268 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bff4b700 1 -- 172.21.15.167:0/4241437686 learned_addr learned my addr 172.21.15.167:0/4241437686 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.258+0000 7fb3bff4b700 1 -- 172.21.15.167:0/4241437686 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f302fe000 con 0x564f302ed800 2024-09-13T23:45:17.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3bff4b700 1 --2- 172.21.15.167:0/4241437686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 0x564f305d2b00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x564f30615d70 tx=0x564f30644240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b4d779508ad422 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3ba740700 1 -- 172.21.15.167:0/4241437686 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f30391600 con 0x564f302ed800 2024-09-13T23:45:17.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3ba740700 1 -- 172.21.15.167:0/4241437686 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f303b1c80 con 0x564f302ed800 2024-09-13T23:45:17.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3ba740700 1 -- 172.21.15.167:0/4241437686 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f30391a20 con 0x564f302ed800 2024-09-13T23:45:17.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3c1792f40 1 -- 172.21.15.167:0/4241437686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 msgr2=0x564f305d2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/4241437686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 0x564f305d2b00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x564f30615d70 tx=0x564f30644240 comp rx=0 tx=0).stop 2024-09-13T23:45:17.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3c1792f40 1 -- 172.21.15.167:0/4241437686 shutdown_connections 2024-09-13T23:45:17.269 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/4241437686 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 0x564f305d2b00 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3c1792f40 1 -- 172.21.15.167:0/4241437686 >> 172.21.15.167:0/4241437686 conn(0x564f302ed400 msgr2=0x564f303ed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3c1792f40 1 -- 172.21.15.167:0/4241437686 shutdown_connections 2024-09-13T23:45:17.270 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3c1792f40 1 -- 172.21.15.167:0/4241437686 wait complete. 2024-09-13T23:45:17.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3c1792f40 1 Processor -- start 2024-09-13T23:45:17.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3c1792f40 1 -- start start 2024-09-13T23:45:17.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3c1792f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 0x564f305d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3c1792f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f30312640 con 0x564f302ed800 2024-09-13T23:45:17.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3bff4b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 0x564f305d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3bff4b700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 0x564f305d2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43186/0 (socket says 172.21.15.167:43186) 2024-09-13T23:45:17.271 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.262+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 learned_addr learned my addr 172.21.15.167:0/548346198 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f302feea0 con 0x564f302ed800 2024-09-13T23:45:17.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3bff4b700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 0x564f305d2b00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x564f306a6c60 tx=0x564f30644d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3b8f3d700 1 -- 172.21.15.167:0/548346198 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f30391600 con 0x564f302ed800 2024-09-13T23:45:17.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3b8f3d700 1 -- 172.21.15.167:0/548346198 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f306b0000 con 0x564f302ed800 2024-09-13T23:45:17.272 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564f302fe820 con 0x564f302ed800 2024-09-13T23:45:17.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3b8f3d700 1 -- 172.21.15.167:0/548346198 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f30391a20 con 0x564f302ed800 2024-09-13T23:45:17.273 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f302fe9c0 con 0x564f302ed800 2024-09-13T23:45:17.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3b8f3d700 1 -- 172.21.15.167:0/548346198 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x564f303e6dc0 con 0x564f302ed800 2024-09-13T23:45:17.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3c1792f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3b8f3d700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f30685000 0x564f305d3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.274 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3b8f3d700 1 -- 172.21.15.167:0/548346198 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x564f30353340 con 0x564f302ed800 2024-09-13T23:45:17.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3bf74a700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f30685000 0x564f305d3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564f30685400 0x564f305d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.266+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f30685800 con 0x564f30685400 2024-09-13T23:45:17.275 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c074c700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564f30685400 0x564f305d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c074c700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564f30685400 0x564f305d3b80 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3bf74a700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f30685000 0x564f305d3600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x564f30615e90 tx=0x564f303f1860 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.276 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c1792f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:17.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c1792f40 20 realm 2024-09-13T23:45:17.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c1792f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.277 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f30685c00 con 0x564f30685400 2024-09-13T23:45:17.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c1792f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:17.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c1792f40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:45:17.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c1792f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f30685800 con 0x564f30685400 2024-09-13T23:45:17.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c1792f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:17.278 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c1792f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564f30685c00 0x564f305d4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a000 con 0x564f30685c00 2024-09-13T23:45:17.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3bff4b700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564f30685c00 0x564f305d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.270+0000 7fb3bff4b700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564f30685c00 0x564f305d4100 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3c1792f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:45:17.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3c1792f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.279 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685c00 2024-09-13T23:45:17.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3c1792f40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:45:17.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3c1792f40 20 searching for the correct realm 2024-09-13T23:45:17.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564f3071a800 0x564f305d4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.280 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071ac00 con 0x564f3071a800 2024-09-13T23:45:17.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3bf74a700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564f3071a800 0x564f305d4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3bf74a700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564f3071a800 0x564f305d4680 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.281 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071ac00 con 0x564f30685c00 2024-09-13T23:45:17.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3bff4b700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564f3071a000 0x564f305d4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.282 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a000 2024-09-13T23:45:17.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.274+0000 7fb3c074c700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564f3071a000 0x564f305d4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3c074c700 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564f3071a000 0x564f305d4c00 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.283 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685400 2024-09-13T23:45:17.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.284 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685400 2024-09-13T23:45:17.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a800 2024-09-13T23:45:17.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.285 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685400 2024-09-13T23:45:17.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685c00 2024-09-13T23:45:17.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.286 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.278+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685c00 2024-09-13T23:45:17.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.287 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685400 2024-09-13T23:45:17.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a000 2024-09-13T23:45:17.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.288 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685400 2024-09-13T23:45:17.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a000 2024-09-13T23:45:17.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.289 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685c00 2024-09-13T23:45:17.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.290 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a800 2024-09-13T23:45:17.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.282+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685400 2024-09-13T23:45:17.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.286+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.291 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.286+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685c00 2024-09-13T23:45:17.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.286+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.292 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.286+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a800 2024-09-13T23:45:17.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.286+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.286+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685400 2024-09-13T23:45:17.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.286+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.293 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.286+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a000 2024-09-13T23:45:17.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.286+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.286+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685400 2024-09-13T23:45:17.294 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.286+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.286+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a800 2024-09-13T23:45:17.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.295 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a000 2024-09-13T23:45:17.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685400 2024-09-13T23:45:17.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.296 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a000 2024-09-13T23:45:17.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a800 2024-09-13T23:45:17.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.297 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a000 2024-09-13T23:45:17.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.298 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685c00 2024-09-13T23:45:17.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.290+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685c00 2024-09-13T23:45:17.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.299 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a800 2024-09-13T23:45:17.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.300 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685c00 2024-09-13T23:45:17.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.301 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685c00 2024-09-13T23:45:17.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3c1792f40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:45:17.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3c1792f40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:45:17.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3c1792f40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:45:17.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3c1792f40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:45:17.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3c1792f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.302 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071b000 con 0x564f30685400 2024-09-13T23:45:17.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3c1792f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:17.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3c1792f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.294+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071b400 con 0x564f30685c00 2024-09-13T23:45:17.303 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:45:17.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071b800 con 0x564f30685c00 2024-09-13T23:45:17.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.304 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:45:17.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 zone default found 2024-09-13T23:45:17.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 4 Realm: () 2024-09-13T23:45:17.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:45:17.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:45:17.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:45:17.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 zonegroup default 2024-09-13T23:45:17.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.305 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071bc00 con 0x564f3071a000 2024-09-13T23:45:17.306 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071b000 con 0x564f3071a800 2024-09-13T23:45:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071bc00 con 0x564f3071a800 2024-09-13T23:45:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:17.307 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 started sync module instance, tier type = 2024-09-13T23:45:17.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:45:17.308 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.298+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f30685800 con 0x564f30685c00 2024-09-13T23:45:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.302+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.309 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.302+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94898113780480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071ac00 con 0x564f30685c00 2024-09-13T23:45:17.310 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.302+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f30685400 2024-09-13T23:45:17.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.306+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.306+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94898113781632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071b400 con 0x564f30685400 2024-09-13T23:45:17.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.306+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071b800 con 0x564f30685c00 2024-09-13T23:45:17.312 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.306+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94898113780480] v35'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.306+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94898113781632] v35'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.310+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.310+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94898113782784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071b000 con 0x564f30685c00 2024-09-13T23:45:17.315 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.310+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071bc00 con 0x564f3071a000 2024-09-13T23:45:17.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.310+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.310+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94898113783936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f30684800 con 0x564f3071a000 2024-09-13T23:45:17.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.310+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f30685800 con 0x564f30685c00 2024-09-13T23:45:17.318 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.310+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94898113782784] v35'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.319 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.314+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94898113783936] v35'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.320 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.314+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.314+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94898113785088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f30684800 con 0x564f30685c00 2024-09-13T23:45:17.321 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.314+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a800 2024-09-13T23:45:17.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.314+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.314+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94898113786240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071b400 con 0x564f3071a800 2024-09-13T23:45:17.322 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.314+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071bc00 con 0x564f3071a800 2024-09-13T23:45:17.323 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.314+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94898113785088] v35'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94898113786240] v35'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.324 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94898113787392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071a400 con 0x564f3071a800 2024-09-13T23:45:17.325 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071b400 con 0x564f3071a000 2024-09-13T23:45:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94898113787392] v35'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'237 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94898113788544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f3071bc00 con 0x564f3071a000 2024-09-13T23:45:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3c1792f40 20 add_watcher() i=0 2024-09-13T23:45:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3c1792f40 20 add_watcher() i=1 2024-09-13T23:45:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3c1792f40 20 add_watcher() i=2 2024-09-13T23:45:17.327 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3c1792f40 20 add_watcher() i=3 2024-09-13T23:45:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3c1792f40 20 add_watcher() i=4 2024-09-13T23:45:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3c1792f40 20 add_watcher() i=5 2024-09-13T23:45:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.318+0000 7fb3c1792f40 20 add_watcher() i=6 2024-09-13T23:45:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.322+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94898113788544] v35'238 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.328 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.322+0000 7fb3c1792f40 20 add_watcher() i=7 2024-09-13T23:45:17.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.322+0000 7fb3c1792f40 2 all 8 watchers are set, enabling cache 2024-09-13T23:45:17.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.322+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f30736000 con 0x564f30685400 2024-09-13T23:45:17.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.322+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.329 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.322+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94898115231744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f30736400 con 0x564f30685400 2024-09-13T23:45:17.331 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94898115231744] v35'237 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3a6f19700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:45:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3c1792f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:45:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3c1792f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:45:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3c1792f40 1 Processor -- start 2024-09-13T23:45:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3c1792f40 1 -- start start 2024-09-13T23:45:17.332 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3c1792f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 0x564f305d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3c1792f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f30312780 con 0x564f30736c00 2024-09-13T23:45:17.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3bf74a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 0x564f305d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3bf74a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 0x564f305d5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43190/0 (socket says 172.21.15.167:43190) 2024-09-13T23:45:17.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3bf74a700 1 -- 172.21.15.167:0/1795693357 learned_addr learned my addr 172.21.15.167:0/1795693357 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.333 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3bf74a700 1 -- 172.21.15.167:0/1795693357 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f302feb60 con 0x564f30736c00 2024-09-13T23:45:17.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3bf74a700 1 --2- 172.21.15.167:0/1795693357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 0x564f305d5180 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x564f306c1b90 tx=0x564f307143c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876f8040b4de6a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3a6718700 1 -- 172.21.15.167:0/1795693357 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f30391760 con 0x564f30736c00 2024-09-13T23:45:17.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3a6718700 1 -- 172.21.15.167:0/1795693357 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f306b0a80 con 0x564f30736c00 2024-09-13T23:45:17.334 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.326+0000 7fb3a6718700 1 -- 172.21.15.167:0/1795693357 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f30738160 con 0x564f30736c00 2024-09-13T23:45:17.335 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 -- 172.21.15.167:0/1795693357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 msgr2=0x564f305d5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/1795693357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 0x564f305d5180 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x564f306c1b90 tx=0x564f307143c0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 -- 172.21.15.167:0/1795693357 shutdown_connections 2024-09-13T23:45:17.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/1795693357 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 0x564f305d5180 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 -- 172.21.15.167:0/1795693357 >> 172.21.15.167:0/1795693357 conn(0x564f30736800 msgr2=0x564f303ee000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 -- 172.21.15.167:0/1795693357 shutdown_connections 2024-09-13T23:45:17.336 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 -- 172.21.15.167:0/1795693357 wait complete. 2024-09-13T23:45:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 Processor -- start 2024-09-13T23:45:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 -- start start 2024-09-13T23:45:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 0x564f305d5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3037a000 con 0x564f30736c00 2024-09-13T23:45:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3bf74a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 0x564f305d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3bf74a700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 0x564f305d5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43202/0 (socket says 172.21.15.167:43202) 2024-09-13T23:45:17.337 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3bf74a700 1 -- 172.21.15.167:0/526962413 learned_addr learned my addr 172.21.15.167:0/526962413 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3bf74a700 1 -- 172.21.15.167:0/526962413 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f302fed00 con 0x564f30736c00 2024-09-13T23:45:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3bf74a700 1 --2- 172.21.15.167:0/526962413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 0x564f305d5180 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x564f30744120 tx=0x564f30714000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3a4f15700 1 -- 172.21.15.167:0/526962413 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f307382c0 con 0x564f30736c00 2024-09-13T23:45:17.338 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3a4f15700 1 -- 172.21.15.167:0/526962413 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564f306b0780 con 0x564f30736c00 2024-09-13T23:45:17.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 -- 172.21.15.167:0/526962413 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564f302fe340 con 0x564f30736c00 2024-09-13T23:45:17.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3a4f15700 1 -- 172.21.15.167:0/526962413 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x564f30738420 con 0x564f30736c00 2024-09-13T23:45:17.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.330+0000 7fb3c1792f40 1 -- 172.21.15.167:0/526962413 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f302ffd40 con 0x564f30736c00 2024-09-13T23:45:17.339 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.334+0000 7fb3a4f15700 1 -- 172.21.15.167:0/526962413 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x564f303e78c0 con 0x564f30736c00 2024-09-13T23:45:17.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.334+0000 7fb3a4f15700 1 --2- 172.21.15.167:0/526962413 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f307c0000 0x564f307c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.334+0000 7fb3a4f15700 1 -- 172.21.15.167:0/526962413 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x564f303536c0 con 0x564f30736c00 2024-09-13T23:45:17.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.334+0000 7fb3c1792f40 5 note: GC not initialized 2024-09-13T23:45:17.340 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.334+0000 7fb3c074c700 1 --2- 172.21.15.167:0/526962413 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f307c0000 0x564f307c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.341 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.334+0000 7fb3c074c700 1 --2- 172.21.15.167:0/526962413 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f307c0000 0x564f307c6000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x564f306a73b0 tx=0x564f30714240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.334+0000 7fb3a2710700 20 reqs_thread_entry: start 2024-09-13T23:45:17.342 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.334+0000 7fb3c1792f40 20 init_complete bucket index max shards: 11 2024-09-13T23:45:17.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.334+0000 7fb3c1792f40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:45:17.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.334+0000 7fb3a070c700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:45:47 2024 2024-09-13T23:45:17.343 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:45:17.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.334+0000 7fb39feea700 20 reqs_thread_entry: start 2024-09-13T23:45:17.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.334+0000 7fb3c1792f40 20 rados->read ofs=0 len=0 2024-09-13T23:45:17.343 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.338+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f30737800 con 0x564f30685400 2024-09-13T23:45:17.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.338+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.344 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.338+0000 7fb3c1792f40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:17.344 INFO:teuthology.orchestra.run.smithi167.stderr:could not fetch user info: no user info saved 2024-09-13T23:45:17.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.338+0000 7fb3a6f19700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:45:17.345 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.338+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94898115231744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f30737c00 con 0x564f30685400 2024-09-13T23:45:17.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.342+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94898115231744] v35'238 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.348 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.342+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94898113780480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f307c0400 con 0x564f30685c00 2024-09-13T23:45:17.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.342+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94898113780480] v35'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.350 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.342+0000 7fb3c1792f40 20 remove_watcher() i=0 2024-09-13T23:45:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.342+0000 7fb3c1792f40 2 removed watcher, disabling cache 2024-09-13T23:45:17.351 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.342+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94898113781632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f307c0800 con 0x564f30685400 2024-09-13T23:45:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.346+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94898113781632] v35'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f30685400 2024-09-13T23:45:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.346+0000 7fb3c1792f40 20 remove_watcher() i=1 2024-09-13T23:45:17.353 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.346+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94898113782784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f307c0c00 con 0x564f30685c00 2024-09-13T23:45:17.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.350+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94898113782784] v35'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.350+0000 7fb3c1792f40 20 remove_watcher() i=2 2024-09-13T23:45:17.355 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.350+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94898113783936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f307c1000 con 0x564f3071a000 2024-09-13T23:45:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.350+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94898113783936] v35'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.350+0000 7fb3c1792f40 20 remove_watcher() i=3 2024-09-13T23:45:17.357 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.350+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94898113785088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f307c1400 con 0x564f30685c00 2024-09-13T23:45:17.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.354+0000 7fb3bff4b700 1 -- 172.21.15.167:0/548346198 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94898113785088] v35'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713200 con 0x564f30685c00 2024-09-13T23:45:17.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.354+0000 7fb3c1792f40 20 remove_watcher() i=4 2024-09-13T23:45:17.360 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.354+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94898113786240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f307c1800 con 0x564f3071a800 2024-09-13T23:45:17.362 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.354+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94898113786240] v35'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.354+0000 7fb3c1792f40 20 remove_watcher() i=5 2024-09-13T23:45:17.363 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.354+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94898113787392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f307c1c00 con 0x564f3071a800 2024-09-13T23:45:17.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.358+0000 7fb3bf74a700 1 -- 172.21.15.167:0/548346198 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94898113787392] v35'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30713440 con 0x564f3071a800 2024-09-13T23:45:17.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.358+0000 7fb3c1792f40 20 remove_watcher() i=6 2024-09-13T23:45:17.365 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.358+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94898113788544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x564f30736000 con 0x564f3071a000 2024-09-13T23:45:17.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94898113788544] v35'239 uv237 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x564f30712b40 con 0x564f3071a000 2024-09-13T23:45:17.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 20 remove_watcher() i=7 2024-09-13T23:45:17.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564f30685c00 msgr2=0x564f305d4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x564f30685c00 0x564f305d4100 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.368 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564f30685400 msgr2=0x564f305d3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x564f30685400 0x564f305d3b80 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564f3071a000 msgr2=0x564f305d4c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x564f3071a000 0x564f305d4c00 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564f3071a800 msgr2=0x564f305d4680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x564f3071a800 0x564f305d4680 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f30685000 msgr2=0x564f305d3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f30685000 0x564f305d3600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x564f30615e90 tx=0x564f303f1860 comp rx=0 tx=0).stop 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c074c700 1 -- 172.21.15.167:0/548346198 reap_dead start 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 msgr2=0x564f305d2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 0x564f305d2b00 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x564f306a6c60 tx=0x564f30644d80 comp rx=0 tx=0).stop 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 shutdown_connections 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/548346198 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed800 0x564f305d2b00 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.369 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 >> 172.21.15.167:0/548346198 conn(0x564f302ed400 msgr2=0x564f303ed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 shutdown_connections 2024-09-13T23:45:17.370 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/548346198 wait complete. 2024-09-13T23:45:17.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/526962413 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f307c0000 msgr2=0x564f307c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/526962413 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f307c0000 0x564f307c6000 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x564f306a73b0 tx=0x564f30714240 comp rx=0 tx=0).stop 2024-09-13T23:45:17.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/526962413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 msgr2=0x564f305d5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/526962413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 0x564f305d5180 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x564f30744120 tx=0x564f30714000 comp rx=0 tx=0).stop 2024-09-13T23:45:17.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/526962413 shutdown_connections 2024-09-13T23:45:17.371 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/526962413 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f307c0000 0x564f307c6000 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/526962413 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f30736c00 0x564f305d5180 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/526962413 >> 172.21.15.167:0/526962413 conn(0x564f30736800 msgr2=0x564f303ee000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/526962413 shutdown_connections 2024-09-13T23:45:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/526962413 wait complete. 2024-09-13T23:45:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/699787343 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f30684000 msgr2=0x564f305d3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/699787343 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f30684000 0x564f305d3080 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x564f305cd920 tx=0x564f303f1800 comp rx=0 tx=0).stop 2024-09-13T23:45:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/699787343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 msgr2=0x564f305d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/699787343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 0x564f305d2580 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x564f310b7e00 tx=0x564f303f1920 comp rx=0 tx=0).stop 2024-09-13T23:45:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/699787343 shutdown_connections 2024-09-13T23:45:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/699787343 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x564f30684000 0x564f305d3080 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 --2- 172.21.15.167:0/699787343 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x564f302ed000 0x564f305d2580 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.372 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/699787343 >> 172.21.15.167:0/699787343 conn(0x564f302ecc00 msgr2=0x564f303ed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/699787343 shutdown_connections 2024-09-13T23:45:17.373 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.362+0000 7fb3c1792f40 1 -- 172.21.15.167:0/699787343 wait complete. 2024-09-13T23:45:17.375 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-13T23:45:17.375 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get', '--rgw-zone', 'default'] 2024-09-13T23:45:17.375 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get', '--rgw-zone', 'default'] 2024-09-13T23:45:17.375 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get --rgw-zone default 2024-09-13T23:45:17.417 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:45:17.417 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:45:17.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.410+0000 7f1bcbd63f40 1 Processor -- start 2024-09-13T23:45:17.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.410+0000 7f1bcbd63f40 1 -- start start 2024-09-13T23:45:17.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.410+0000 7f1bcbd63f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f61e4ed000 0x55f61e7d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.410+0000 7f1bcbd63f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55f61e5b1500 con 0x55f61e4ed400 2024-09-13T23:45:17.418 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.410+0000 7f1bcbd63f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55f61e57aa00 con 0x55f61e4ed000 2024-09-13T23:45:17.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.410+0000 7f1bca51c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f61e4ed000 0x55f61e7d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.410+0000 7f1bca51c700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55f61e4ed000 0x55f61e7d2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43216/0 (socket says 172.21.15.167:43216) 2024-09-13T23:45:17.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.410+0000 7f1bca51c700 1 -- 172.21.15.167:0/2933045991 learned_addr learned my addr 172.21.15.167:0/2933045991 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.410+0000 7f1bc9d1b700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55f61e4ed400 0x55f61e5ec800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:38786/0 (socket says 172.21.15.167:38786) 2024-09-13T23:45:17.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bca51c700 1 -- 172.21.15.167:0/2933045991 >> v1:172.21.15.12:6789/0 conn(0x55f61e4ed400 legacy=0x55f61e5ec800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.419 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bca51c700 1 -- 172.21.15.167:0/2933045991 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f61e4ffd40 con 0x55f61e4ed000 2024-09-13T23:45:17.420 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bca51c700 1 --2- 172.21.15.167:0/2933045991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x55f61e7cda70 tx=0x55f61e5f1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f94326456c8516 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bc951a700 1 -- 172.21.15.167:0/2933045991 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e5914a0 con 0x55f61e4ed000 2024-09-13T23:45:17.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bc951a700 1 -- 172.21.15.167:0/2933045991 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f61e5b1800 con 0x55f61e4ed000 2024-09-13T23:45:17.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bc951a700 1 -- 172.21.15.167:0/2933045991 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e591600 con 0x55f61e4ed000 2024-09-13T23:45:17.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/2933045991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 msgr2=0x55f61e7d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/2933045991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x55f61e7cda70 tx=0x55f61e5f1920 comp rx=0 tx=0).stop 2024-09-13T23:45:17.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/2933045991 shutdown_connections 2024-09-13T23:45:17.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/2933045991 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.421 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/2933045991 >> 172.21.15.167:0/2933045991 conn(0x55f61e4ecc00 msgr2=0x55f61e5ec000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/2933045991 shutdown_connections 2024-09-13T23:45:17.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/2933045991 wait complete. 2024-09-13T23:45:17.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bcbd63f40 1 Processor -- start 2024-09-13T23:45:17.422 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bcbd63f40 1 -- start start 2024-09-13T23:45:17.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bcbd63f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bcbd63f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f61e57a500 con 0x55f61e4ed000 2024-09-13T23:45:17.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bca51c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bca51c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43220/0 (socket says 172.21.15.167:43220) 2024-09-13T23:45:17.423 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.414+0000 7f1bca51c700 1 -- 172.21.15.167:0/3716750850 learned_addr learned my addr 172.21.15.167:0/3716750850 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bca51c700 1 -- 172.21.15.167:0/3716750850 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f61e4ffba0 con 0x55f61e4ed000 2024-09-13T23:45:17.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bca51c700 1 --2- 172.21.15.167:0/3716750850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55f61e7ccc30 tx=0x55f61e5f1800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d268f4a18788c380 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bc8d19700 1 -- 172.21.15.167:0/3716750850 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e591760 con 0x55f61e4ed000 2024-09-13T23:45:17.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bc8d19700 1 -- 172.21.15.167:0/3716750850 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f61e5b1b00 con 0x55f61e4ed000 2024-09-13T23:45:17.424 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bc8d19700 1 -- 172.21.15.167:0/3716750850 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e5914a0 con 0x55f61e4ed000 2024-09-13T23:45:17.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/3716750850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 msgr2=0x55f61e7d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/3716750850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55f61e7ccc30 tx=0x55f61e5f1800 comp rx=0 tx=0).stop 2024-09-13T23:45:17.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/3716750850 shutdown_connections 2024-09-13T23:45:17.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/3716750850 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/3716750850 >> 172.21.15.167:0/3716750850 conn(0x55f61e4ecc00 msgr2=0x55f61e5ed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.425 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/3716750850 shutdown_connections 2024-09-13T23:45:17.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/3716750850 wait complete. 2024-09-13T23:45:17.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bcbd63f40 1 Processor -- start 2024-09-13T23:45:17.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bcbd63f40 1 -- start start 2024-09-13T23:45:17.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bcbd63f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.426 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bcbd63f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f61e57a3c0 con 0x55f61e4ed000 2024-09-13T23:45:17.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bca51c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bca51c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43222/0 (socket says 172.21.15.167:43222) 2024-09-13T23:45:17.427 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.418+0000 7f1bca51c700 1 -- 172.21.15.167:0/523290996 learned_addr learned my addr 172.21.15.167:0/523290996 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bca51c700 1 -- 172.21.15.167:0/523290996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f61e4ffa00 con 0x55f61e4ed000 2024-09-13T23:45:17.428 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bca51c700 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55f61e649e90 tx=0x55f61e5f1920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bc7516700 1 -- 172.21.15.167:0/523290996 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e591600 con 0x55f61e4ed000 2024-09-13T23:45:17.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bc7516700 1 -- 172.21.15.167:0/523290996 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f61e5b1c80 con 0x55f61e4ed000 2024-09-13T23:45:17.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/523290996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f61e4ff860 con 0x55f61e4ed000 2024-09-13T23:45:17.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bc7516700 1 -- 172.21.15.167:0/523290996 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e5918c0 con 0x55f61e4ed000 2024-09-13T23:45:17.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/523290996 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f61e4ff6c0 con 0x55f61e4ed000 2024-09-13T23:45:17.429 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bcbd63f40 1 Processor -- start 2024-09-13T23:45:17.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bcbd63f40 1 -- start start 2024-09-13T23:45:17.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bcbd63f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 0x55f61e7d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.430 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bcbd63f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f61e57a000 con 0x55f61e4ed800 2024-09-13T23:45:17.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bc7516700 1 -- 172.21.15.167:0/523290996 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f61e5e6b00 con 0x55f61e4ed000 2024-09-13T23:45:17.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bcad1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 0x55f61e7d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bc7516700 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e892000 0x55f61e7d3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bcad1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 0x55f61e7d2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43236/0 (socket says 172.21.15.167:43236) 2024-09-13T23:45:17.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bc7516700 1 -- 172.21.15.167:0/523290996 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f61e553180 con 0x55f61e4ed000 2024-09-13T23:45:17.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.422+0000 7f1bcad1d700 1 -- 172.21.15.167:0/3657767632 learned_addr learned my addr 172.21.15.167:0/3657767632 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.431 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bca51c700 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e892000 0x55f61e7d3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcad1d700 1 -- 172.21.15.167:0/3657767632 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f61e4fe000 con 0x55f61e4ed800 2024-09-13T23:45:17.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bca51c700 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e892000 0x55f61e7d3080 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55f61e819500 tx=0x55f61e854b40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcad1d700 1 --2- 172.21.15.167:0/3657767632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 0x55f61e7d2b00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x55f61e649e30 tx=0x55f61e854e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee7099408b0d850 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bc4d11700 1 -- 172.21.15.167:0/3657767632 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e591600 con 0x55f61e4ed800 2024-09-13T23:45:17.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bc4d11700 1 -- 172.21.15.167:0/3657767632 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f61e89c000 con 0x55f61e4ed800 2024-09-13T23:45:17.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bc4d11700 1 -- 172.21.15.167:0/3657767632 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e5918c0 con 0x55f61e4ed800 2024-09-13T23:45:17.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/3657767632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 msgr2=0x55f61e7d2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/3657767632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 0x55f61e7d2b00 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x55f61e649e30 tx=0x55f61e854e40 comp rx=0 tx=0).stop 2024-09-13T23:45:17.433 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/3657767632 shutdown_connections 2024-09-13T23:45:17.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/3657767632 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 0x55f61e7d2b00 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/3657767632 >> 172.21.15.167:0/3657767632 conn(0x55f61e4ed400 msgr2=0x55f61e5ed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/3657767632 shutdown_connections 2024-09-13T23:45:17.434 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/3657767632 wait complete. 2024-09-13T23:45:17.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcbd63f40 1 Processor -- start 2024-09-13T23:45:17.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcbd63f40 1 -- start start 2024-09-13T23:45:17.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcbd63f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 0x55f61e7d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcbd63f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f61e512640 con 0x55f61e4ed800 2024-09-13T23:45:17.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcad1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 0x55f61e7d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcad1d700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 0x55f61e7d2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43252/0 (socket says 172.21.15.167:43252) 2024-09-13T23:45:17.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcad1d700 1 -- 172.21.15.167:0/1432202719 learned_addr learned my addr 172.21.15.167:0/1432202719 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.435 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.426+0000 7f1bcad1d700 1 -- 172.21.15.167:0/1432202719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f61e4feea0 con 0x55f61e4ed800 2024-09-13T23:45:17.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bcad1d700 1 --2- 172.21.15.167:0/1432202719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 0x55f61e7d2b00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x55f61e818840 tx=0x55f61e8545a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bc350e700 1 -- 172.21.15.167:0/1432202719 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e591760 con 0x55f61e4ed800 2024-09-13T23:45:17.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bc350e700 1 -- 172.21.15.167:0/1432202719 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f61e89c180 con 0x55f61e4ed800 2024-09-13T23:45:17.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/1432202719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f61e4fe820 con 0x55f61e4ed800 2024-09-13T23:45:17.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bc350e700 1 -- 172.21.15.167:0/1432202719 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e591600 con 0x55f61e4ed800 2024-09-13T23:45:17.436 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/1432202719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f61e4fe9c0 con 0x55f61e4ed800 2024-09-13T23:45:17.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bc350e700 1 -- 172.21.15.167:0/1432202719 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f61e5e6dc0 con 0x55f61e4ed800 2024-09-13T23:45:17.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bc350e700 1 --2- 172.21.15.167:0/1432202719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e892c00 0x55f61e7d3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bc350e700 1 -- 172.21.15.167:0/1432202719 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f61e553340 con 0x55f61e4ed800 2024-09-13T23:45:17.437 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bc9d1b700 1 --2- 172.21.15.167:0/1432202719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e892c00 0x55f61e7d3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bc9d1b700 1 --2- 172.21.15.167:0/1432202719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e892c00 0x55f61e7d3600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55f61e7cdf80 tx=0x55f61e9360c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bcbd63f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:45:17.438 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bcbd63f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:45:17.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.430+0000 7f1bcbd63f40 1 Processor -- start 2024-09-13T23:45:17.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 -- start start 2024-09-13T23:45:17.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 0x55f61e7d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.439 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f61e5128c0 con 0x55f61e893400 2024-09-13T23:45:17.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bca51c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 0x55f61e7d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bca51c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 0x55f61e7d3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43262/0 (socket says 172.21.15.167:43262) 2024-09-13T23:45:17.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bca51c700 1 -- 172.21.15.167:0/248524958 learned_addr learned my addr 172.21.15.167:0/248524958 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bca51c700 1 -- 172.21.15.167:0/248524958 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f61e4feb60 con 0x55f61e893400 2024-09-13T23:45:17.440 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bca51c700 1 --2- 172.21.15.167:0/248524958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 0x55f61e7d3b80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55f61e8faf60 tx=0x55f61e854cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c5e64eb014894 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bb14ea700 1 -- 172.21.15.167:0/248524958 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e591760 con 0x55f61e893400 2024-09-13T23:45:17.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bb14ea700 1 -- 172.21.15.167:0/248524958 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f61e89c600 con 0x55f61e893400 2024-09-13T23:45:17.441 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bb14ea700 1 -- 172.21.15.167:0/248524958 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e591600 con 0x55f61e893400 2024-09-13T23:45:17.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/248524958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 msgr2=0x55f61e7d3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.442 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/248524958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 0x55f61e7d3b80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x55f61e8faf60 tx=0x55f61e854cc0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/248524958 shutdown_connections 2024-09-13T23:45:17.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/248524958 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 0x55f61e7d3b80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/248524958 >> 172.21.15.167:0/248524958 conn(0x55f61e893000 msgr2=0x55f61e5ee000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/248524958 shutdown_connections 2024-09-13T23:45:17.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/248524958 wait complete. 2024-09-13T23:45:17.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 Processor -- start 2024-09-13T23:45:17.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 -- start start 2024-09-13T23:45:17.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 0x55f61e7d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.443 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bcbd63f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f61e512f00 con 0x55f61e893400 2024-09-13T23:45:17.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.434+0000 7f1bca51c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 0x55f61e7d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.444 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bca51c700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 0x55f61e7d3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43266/0 (socket says 172.21.15.167:43266) 2024-09-13T23:45:17.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bca51c700 1 -- 172.21.15.167:0/349286500 learned_addr learned my addr 172.21.15.167:0/349286500 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bca51c700 1 -- 172.21.15.167:0/349286500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f61e4fed00 con 0x55f61e893400 2024-09-13T23:45:17.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bca51c700 1 --2- 172.21.15.167:0/349286500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 0x55f61e7d3b80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x55f61e8fb140 tx=0x55f61e854de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bafce7700 1 -- 172.21.15.167:0/349286500 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e591a20 con 0x55f61e893400 2024-09-13T23:45:17.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bafce7700 1 -- 172.21.15.167:0/349286500 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f61e89c300 con 0x55f61e893400 2024-09-13T23:45:17.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/349286500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f61e4fe340 con 0x55f61e893400 2024-09-13T23:45:17.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bafce7700 1 -- 172.21.15.167:0/349286500 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55f61e591b80 con 0x55f61e893400 2024-09-13T23:45:17.445 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/349286500 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f61e95c000 con 0x55f61e893400 2024-09-13T23:45:17.446 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bafce7700 1 -- 172.21.15.167:0/349286500 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55f61e5e7b80 con 0x55f61e893400 2024-09-13T23:45:17.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1badce3700 20 reqs_thread_entry: start 2024-09-13T23:45:17.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f61e952800 0x55f61e7d4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bafce7700 1 --2- 172.21.15.167:0/349286500 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e953c00 0x55f61e7d4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/523290996 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f61e952c00 con 0x55f61e952800 2024-09-13T23:45:17.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bafce7700 1 -- 172.21.15.167:0/349286500 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55f61e553500 con 0x55f61e893400 2024-09-13T23:45:17.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.438+0000 7f1bcad1d700 1 --2- 172.21.15.167:0/349286500 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e953c00 0x55f61e7d4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.447 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bc9d1b700 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f61e952800 0x55f61e7d4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bcad1d700 1 --2- 172.21.15.167:0/349286500 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e953c00 0x55f61e7d4c00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55f61e86df50 tx=0x55f61e855500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bc9d1b700 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f61e952800 0x55f61e7d4100 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bc9d1b700 1 -- 172.21.15.167:0/523290996 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55f61e9ca240 con 0x55f61e952800 2024-09-13T23:45:17.448 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/523290996 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:2 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55f61e953000 con 0x55f61e952800 2024-09-13T23:45:17.449 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bc9d1b700 1 -- 172.21.15.167:0/523290996 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(2 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x55f61e9ca240 con 0x55f61e952800 2024-09-13T23:45:17.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/1432202719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e892c00 msgr2=0x55f61e7d3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/1432202719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e892c00 0x55f61e7d3600 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x55f61e7cdf80 tx=0x55f61e9360c0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/1432202719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 msgr2=0x55f61e7d2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/1432202719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 0x55f61e7d2b00 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x55f61e818840 tx=0x55f61e8545a0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/1432202719 shutdown_connections 2024-09-13T23:45:17.451 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/1432202719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e892c00 0x55f61e7d3600 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/1432202719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed800 0x55f61e7d2b00 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.442+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/1432202719 >> 172.21.15.167:0/1432202719 conn(0x55f61e4ed400 msgr2=0x55f61e5ed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/1432202719 shutdown_connections 2024-09-13T23:45:17.452 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/1432202719 wait complete. 2024-09-13T23:45:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/349286500 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e953c00 msgr2=0x55f61e7d4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/349286500 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e953c00 0x55f61e7d4c00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x55f61e86df50 tx=0x55f61e855500 comp rx=0 tx=0).stop 2024-09-13T23:45:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/349286500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 msgr2=0x55f61e7d3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/349286500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 0x55f61e7d3b80 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x55f61e8fb140 tx=0x55f61e854de0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/349286500 shutdown_connections 2024-09-13T23:45:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/349286500 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e953c00 0x55f61e7d4c00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/349286500 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e893400 0x55f61e7d3b80 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/349286500 >> 172.21.15.167:0/349286500 conn(0x55f61e893000 msgr2=0x55f61e5ee000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/349286500 shutdown_connections 2024-09-13T23:45:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/349286500 wait complete. 2024-09-13T23:45:17.454 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/523290996 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f61e952800 msgr2=0x55f61e7d4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f61e952800 0x55f61e7d4100 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/523290996 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e892000 msgr2=0x55f61e7d3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e892000 0x55f61e7d3080 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55f61e819500 tx=0x55f61e854b40 comp rx=0 tx=0).stop 2024-09-13T23:45:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/523290996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 msgr2=0x55f61e7d2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x55f61e649e90 tx=0x55f61e5f1920 comp rx=0 tx=0).stop 2024-09-13T23:45:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/523290996 shutdown_connections 2024-09-13T23:45:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55f61e952800 0x55f61e7d4100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55f61e892000 0x55f61e7d3080 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 --2- 172.21.15.167:0/523290996 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55f61e4ed000 0x55f61e7d2580 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/523290996 >> 172.21.15.167:0/523290996 conn(0x55f61e4ecc00 msgr2=0x55f61e5ed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/523290996 shutdown_connections 2024-09-13T23:45:17.455 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.446+0000 7f1bcbd63f40 1 -- 172.21.15.167:0/523290996 wait complete. 2024-09-13T23:45:17.456 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:45:17.456 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "9144208c-3c85-4fb6-9007-2693d5a930f0", 2024-09-13T23:45:17.456 INFO:teuthology.orchestra.run.smithi167.stdout: "name": "default", 2024-09-13T23:45:17.456 INFO:teuthology.orchestra.run.smithi167.stdout: "domain_root": "default.rgw.meta:root", 2024-09-13T23:45:17.456 INFO:teuthology.orchestra.run.smithi167.stdout: "control_pool": "default.rgw.control", 2024-09-13T23:45:17.456 INFO:teuthology.orchestra.run.smithi167.stdout: "gc_pool": "default.rgw.log:gc", 2024-09-13T23:45:17.456 INFO:teuthology.orchestra.run.smithi167.stdout: "lc_pool": "default.rgw.log:lc", 2024-09-13T23:45:17.456 INFO:teuthology.orchestra.run.smithi167.stdout: "log_pool": "default.rgw.log", 2024-09-13T23:45:17.456 INFO:teuthology.orchestra.run.smithi167.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: "roles_pool": "default.rgw.meta:roles", 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: "otp_pool": "default.rgw.otp", 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: "system_key": { 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "", 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "" 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.457 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_pools": [ 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "default-placement", 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: "val": { 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": "default.rgw.buckets.index", 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: "storage_classes": { 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: "STANDARD": { 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": "default.rgw.buckets.data" 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: "index_type": 0, 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: "inline_data": true 2024-09-13T23:45:17.458 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.459 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.459 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:45:17.459 INFO:teuthology.orchestra.run.smithi167.stdout: "realm_id": "", 2024-09-13T23:45:17.459 INFO:teuthology.orchestra.run.smithi167.stdout: "notif_pool": "default.rgw.log:notif" 2024-09-13T23:45:17.459 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:45:17.460 DEBUG:tasks.util.rgw: json result: {'id': '9144208c-3c85-4fb6-9007-2693d5a930f0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-09-13T23:45:17.460 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'set'] 2024-09-13T23:45:17.460 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'set'] 2024-09-13T23:45:17.460 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone set 2024-09-13T23:45:17.541 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:45:17.541 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:45:17.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.534+0000 7f4c89c27f40 1 Processor -- start 2024-09-13T23:45:17.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.534+0000 7f4c89c27f40 1 -- start start 2024-09-13T23:45:17.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.534+0000 7f4c89c27f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55a49792b400 0x55a497c10580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.534+0000 7f4c89c27f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55a4979ef500 con 0x55a49792b000 2024-09-13T23:45:17.542 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.534+0000 7f4c89c27f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55a4979b8a00 con 0x55a49792b400 2024-09-13T23:45:17.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.534+0000 7f4c87bdf700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55a49792b400 0x55a497c10580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.534+0000 7f4c883e0700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55a49792b000 0x55a497a2a800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:38802/0 (socket says 172.21.15.167:38802) 2024-09-13T23:45:17.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.534+0000 7f4c883e0700 1 -- 172.21.15.167:0/2012779306 learned_addr learned my addr 172.21.15.167:0/2012779306 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:45:17.543 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c87bdf700 1 -- 172.21.15.167:0/2012779306 >> v1:172.21.15.12:6789/0 conn(0x55a49792b000 legacy=0x55a497a2a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c873de700 1 -- 172.21.15.167:0/2012779306 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744384278 0 0) 0x55a497a86000 con 0x55a49792b000 2024-09-13T23:45:17.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c87bdf700 1 -- 172.21.15.167:0/2012779306 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49793dd40 con 0x55a49792b400 2024-09-13T23:45:17.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c87bdf700 1 --2- 172.21.15.167:0/2012779306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b400 0x55a497c10580 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x55a497c0b920 tx=0x55a497a2f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a67c0a859949901 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.544 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c873de700 1 -- 172.21.15.167:0/2012779306 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf4a0 con 0x55a49792b400 2024-09-13T23:45:17.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c873de700 1 -- 172.21.15.167:0/2012779306 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a4979ef800 con 0x55a49792b400 2024-09-13T23:45:17.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c873de700 1 -- 172.21.15.167:0/2012779306 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf600 con 0x55a49792b400 2024-09-13T23:45:17.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c89c27f40 1 -- 172.21.15.167:0/2012779306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b400 msgr2=0x55a497c10580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/2012779306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b400 0x55a497c10580 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x55a497c0b920 tx=0x55a497a2f920 comp rx=0 tx=0).stop 2024-09-13T23:45:17.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c89c27f40 1 -- 172.21.15.167:0/2012779306 shutdown_connections 2024-09-13T23:45:17.545 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/2012779306 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b400 0x55a497c10580 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c89c27f40 1 -- 172.21.15.167:0/2012779306 >> 172.21.15.167:0/2012779306 conn(0x55a49792ac00 msgr2=0x55a497a2a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c89c27f40 1 -- 172.21.15.167:0/2012779306 shutdown_connections 2024-09-13T23:45:17.546 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c89c27f40 1 -- 172.21.15.167:0/2012779306 wait complete. 2024-09-13T23:45:17.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c89c27f40 1 Processor -- start 2024-09-13T23:45:17.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c89c27f40 1 -- start start 2024-09-13T23:45:17.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c89c27f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 0x55a497c10580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c89c27f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4979b8500 con 0x55a49792b000 2024-09-13T23:45:17.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c883e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 0x55a497c10580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c883e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 0x55a497c10580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43284/0 (socket says 172.21.15.167:43284) 2024-09-13T23:45:17.547 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.538+0000 7f4c883e0700 1 -- 172.21.15.167:0/2402313757 learned_addr learned my addr 172.21.15.167:0/2402313757 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c883e0700 1 -- 172.21.15.167:0/2402313757 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49793dba0 con 0x55a49792b000 2024-09-13T23:45:17.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c883e0700 1 --2- 172.21.15.167:0/2402313757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 0x55a497c10580 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x55a4986f7d70 tx=0x55a497a2fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b31bef8f1403c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c86bdd700 1 -- 172.21.15.167:0/2402313757 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf1e0 con 0x55a49792b000 2024-09-13T23:45:17.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c86bdd700 1 -- 172.21.15.167:0/2402313757 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a4979efb00 con 0x55a49792b000 2024-09-13T23:45:17.548 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c86bdd700 1 -- 172.21.15.167:0/2402313757 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf4a0 con 0x55a49792b000 2024-09-13T23:45:17.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c89c27f40 1 -- 172.21.15.167:0/2402313757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 msgr2=0x55a497c10580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/2402313757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 0x55a497c10580 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x55a4986f7d70 tx=0x55a497a2fb60 comp rx=0 tx=0).stop 2024-09-13T23:45:17.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c89c27f40 1 -- 172.21.15.167:0/2402313757 shutdown_connections 2024-09-13T23:45:17.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/2402313757 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 0x55a497c10580 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c89c27f40 1 -- 172.21.15.167:0/2402313757 >> 172.21.15.167:0/2402313757 conn(0x55a49792ac00 msgr2=0x55a497a2b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c89c27f40 1 -- 172.21.15.167:0/2402313757 shutdown_connections 2024-09-13T23:45:17.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c89c27f40 1 -- 172.21.15.167:0/2402313757 wait complete. 2024-09-13T23:45:17.552 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c89c27f40 1 Processor -- start 2024-09-13T23:45:17.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c89c27f40 1 -- start start 2024-09-13T23:45:17.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c89c27f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 0x55a497c10580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c89c27f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4979b83c0 con 0x55a49792b000 2024-09-13T23:45:17.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c883e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 0x55a497c10580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c883e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 0x55a497c10580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43296/0 (socket says 172.21.15.167:43296) 2024-09-13T23:45:17.553 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 learned_addr learned my addr 172.21.15.167:0/248759112 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49793da00 con 0x55a49792b000 2024-09-13T23:45:17.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c883e0700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 0x55a497c10580 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x55a4986f7e60 tx=0x55a497a2fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.542+0000 7f4c853da700 1 -- 172.21.15.167:0/248759112 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf600 con 0x55a49792b000 2024-09-13T23:45:17.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c853da700 1 -- 172.21.15.167:0/248759112 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a4979efc80 con 0x55a49792b000 2024-09-13T23:45:17.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a49793d860 con 0x55a49792b000 2024-09-13T23:45:17.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c853da700 1 -- 172.21.15.167:0/248759112 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf1e0 con 0x55a49792b000 2024-09-13T23:45:17.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a49793d6c0 con 0x55a49792b000 2024-09-13T23:45:17.555 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c853da700 1 -- 172.21.15.167:0/248759112 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55a497a24b00 con 0x55a49792b000 2024-09-13T23:45:17.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c853da700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497cd0000 0x55a497c11080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c89c27f40 1 Processor -- start 2024-09-13T23:45:17.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c853da700 1 -- 172.21.15.167:0/248759112 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55a497991180 con 0x55a49792b000 2024-09-13T23:45:17.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c89c27f40 1 -- start start 2024-09-13T23:45:17.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c87bdf700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497cd0000 0x55a497c11080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c89c27f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 0x55a497c10b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c89c27f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4979b8000 con 0x55a49792b800 2024-09-13T23:45:17.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c883e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 0x55a497c10b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c883e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 0x55a497c10b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43312/0 (socket says 172.21.15.167:43312) 2024-09-13T23:45:17.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c883e0700 1 -- 172.21.15.167:0/3088249908 learned_addr learned my addr 172.21.15.167:0/3088249908 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c87bdf700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497cd0000 0x55a497c11080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55a497c0b920 tx=0x55a497a2f800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.556 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c883e0700 1 -- 172.21.15.167:0/3088249908 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49793c000 con 0x55a49792b800 2024-09-13T23:45:17.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c883e0700 1 --2- 172.21.15.167:0/3088249908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 0x55a497c10b00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x55a497c5dd70 tx=0x55a497c8a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551556a4838296ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c82bd5700 1 -- 172.21.15.167:0/3088249908 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf4a0 con 0x55a49792b800 2024-09-13T23:45:17.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c82bd5700 1 -- 172.21.15.167:0/3088249908 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a4979efc80 con 0x55a49792b800 2024-09-13T23:45:17.557 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.546+0000 7f4c82bd5700 1 -- 172.21.15.167:0/3088249908 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf8c0 con 0x55a49792b800 2024-09-13T23:45:17.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 -- 172.21.15.167:0/3088249908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 msgr2=0x55a497c10b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/3088249908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 0x55a497c10b00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x55a497c5dd70 tx=0x55a497c8a240 comp rx=0 tx=0).stop 2024-09-13T23:45:17.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 -- 172.21.15.167:0/3088249908 shutdown_connections 2024-09-13T23:45:17.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/3088249908 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 0x55a497c10b00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 -- 172.21.15.167:0/3088249908 >> 172.21.15.167:0/3088249908 conn(0x55a49792b400 msgr2=0x55a497a2b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 -- 172.21.15.167:0/3088249908 shutdown_connections 2024-09-13T23:45:17.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 -- 172.21.15.167:0/3088249908 wait complete. 2024-09-13T23:45:17.559 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 Processor -- start 2024-09-13T23:45:17.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 -- start start 2024-09-13T23:45:17.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 0x55a497c10b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a497950640 con 0x55a49792b800 2024-09-13T23:45:17.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c883e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 0x55a497c10b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c883e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 0x55a497c10b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43328/0 (socket says 172.21.15.167:43328) 2024-09-13T23:45:17.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c883e0700 1 -- 172.21.15.167:0/135519409 learned_addr learned my addr 172.21.15.167:0/135519409 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c883e0700 1 -- 172.21.15.167:0/135519409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49793cea0 con 0x55a49792b800 2024-09-13T23:45:17.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c883e0700 1 --2- 172.21.15.167:0/135519409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 0x55a497c10b00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x55a497ceec90 tx=0x55a497c8ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c813d2700 1 -- 172.21.15.167:0/135519409 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf4a0 con 0x55a49792b800 2024-09-13T23:45:17.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c813d2700 1 -- 172.21.15.167:0/135519409 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a497cf4000 con 0x55a49792b800 2024-09-13T23:45:17.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c813d2700 1 -- 172.21.15.167:0/135519409 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf8c0 con 0x55a49792b800 2024-09-13T23:45:17.560 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 -- 172.21.15.167:0/135519409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a49793c820 con 0x55a49792b800 2024-09-13T23:45:17.561 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.550+0000 7f4c89c27f40 1 -- 172.21.15.167:0/135519409 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a49793c9c0 con 0x55a49792b800 2024-09-13T23:45:17.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c813d2700 1 -- 172.21.15.167:0/135519409 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55a497a24dc0 con 0x55a49792b800 2024-09-13T23:45:17.562 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c813d2700 1 --2- 172.21.15.167:0/135519409 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497cd1000 0x55a497c11600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c813d2700 1 -- 172.21.15.167:0/135519409 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55a497991340 con 0x55a49792b800 2024-09-13T23:45:17.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c87bdf700 1 --2- 172.21.15.167:0/135519409 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497cd1000 0x55a497c11600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c89c27f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:45:17.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c89c27f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:45:17.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c89c27f40 1 Processor -- start 2024-09-13T23:45:17.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c87bdf700 1 --2- 172.21.15.167:0/135519409 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497cd1000 0x55a497c11600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55a497c5de90 tx=0x55a497a2f860 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c89c27f40 1 -- start start 2024-09-13T23:45:17.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c89c27f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 0x55a497c11b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c89c27f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4979508c0 con 0x55a497cd1800 2024-09-13T23:45:17.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c883e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 0x55a497c11b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c883e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 0x55a497c11b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43342/0 (socket says 172.21.15.167:43342) 2024-09-13T23:45:17.563 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.554+0000 7f4c883e0700 1 -- 172.21.15.167:0/1322619355 learned_addr learned my addr 172.21.15.167:0/1322619355 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c883e0700 1 -- 172.21.15.167:0/1322619355 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49793cb60 con 0x55a497cd1800 2024-09-13T23:45:17.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c883e0700 1 --2- 172.21.15.167:0/1322619355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 0x55a497c11b80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x55a497d043f0 tx=0x55a497c8b320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5855bb35e79620d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c6f3ae700 1 -- 172.21.15.167:0/1322619355 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf600 con 0x55a497cd1800 2024-09-13T23:45:17.564 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c6f3ae700 1 -- 172.21.15.167:0/1322619355 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a497cf4300 con 0x55a497cd1800 2024-09-13T23:45:17.565 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c6f3ae700 1 -- 172.21.15.167:0/1322619355 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf1e0 con 0x55a497cd1800 2024-09-13T23:45:17.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1322619355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 msgr2=0x55a497c11b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/1322619355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 0x55a497c11b80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x55a497d043f0 tx=0x55a497c8b320 comp rx=0 tx=0).stop 2024-09-13T23:45:17.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1322619355 shutdown_connections 2024-09-13T23:45:17.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/1322619355 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 0x55a497c11b80 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1322619355 >> 172.21.15.167:0/1322619355 conn(0x55a497cd1400 msgr2=0x55a497a2c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1322619355 shutdown_connections 2024-09-13T23:45:17.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1322619355 wait complete. 2024-09-13T23:45:17.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c89c27f40 1 Processor -- start 2024-09-13T23:45:17.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c89c27f40 1 -- start start 2024-09-13T23:45:17.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c89c27f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 0x55a497c11b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.566 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c89c27f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a497950f00 con 0x55a497cd1800 2024-09-13T23:45:17.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c883e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 0x55a497c11b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c883e0700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 0x55a497c11b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43344/0 (socket says 172.21.15.167:43344) 2024-09-13T23:45:17.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c883e0700 1 -- 172.21.15.167:0/1164112658 learned_addr learned my addr 172.21.15.167:0/1164112658 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.558+0000 7f4c883e0700 1 -- 172.21.15.167:0/1164112658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a49793cd00 con 0x55a497cd1800 2024-09-13T23:45:17.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c883e0700 1 --2- 172.21.15.167:0/1164112658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 0x55a497c11b80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x55a497d05260 tx=0x55a497a2ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c6dbab700 1 -- 172.21.15.167:0/1164112658 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf600 con 0x55a497cd1800 2024-09-13T23:45:17.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c6dbab700 1 -- 172.21.15.167:0/1164112658 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a497cf4480 con 0x55a497cd1800 2024-09-13T23:45:17.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1164112658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a49793c340 con 0x55a497cd1800 2024-09-13T23:45:17.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c6dbab700 1 -- 172.21.15.167:0/1164112658 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55a4979cf1e0 con 0x55a497cd1800 2024-09-13T23:45:17.568 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1164112658 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a497d66000 con 0x55a497cd1800 2024-09-13T23:45:17.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c6dbab700 1 -- 172.21.15.167:0/1164112658 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55a497a25080 con 0x55a497cd1800 2024-09-13T23:45:17.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c6dbab700 1 --2- 172.21.15.167:0/1164112658 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497d69400 0x55a497c12c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c6dbab700 1 -- 172.21.15.167:0/1164112658 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55a497991500 con 0x55a497cd1800 2024-09-13T23:45:17.569 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c87bdf700 1 --2- 172.21.15.167:0/1164112658 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497d69400 0x55a497c12c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c87bdf700 1 --2- 172.21.15.167:0/1164112658 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497d69400 0x55a497c12c00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55a497d7c900 tx=0x55a497a2f8c0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c6bba7700 20 reqs_thread_entry: start 2024-09-13T23:45:17.570 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a497d68400 0x55a497c12100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d68800 con 0x55a497d68400 2024-09-13T23:45:17.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.562+0000 7f4c88be1700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a497d68400 0x55a497c12100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.571 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.566+0000 7f4c88be1700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a497d68400 0x55a497c12100 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.566+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d68400 2024-09-13T23:45:17.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.566+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a497d68c00 0x55a497c12680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.566+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69000 con 0x55a497d68c00 2024-09-13T23:45:17.572 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.566+0000 7f4c883e0700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a497d68c00 0x55a497c12680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.566+0000 7f4c883e0700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a497d68c00 0x55a497c12680 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.566+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.573 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.566+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:3 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68c00 2024-09-13T23:45:17.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.566+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(3 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.574 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.566+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:4 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69c00 con 0x55a497d68400 2024-09-13T23:45:17.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(4 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d68400 2024-09-13T23:45:17.575 INFO:teuthology.orchestra.run.smithi167.stderr:NOTICE: set zone's realm_id= 2024-09-13T23:45:17.575 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a497d68800 0x55a497c13180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:5 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69c00 con 0x55a497d68800 2024-09-13T23:45:17.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c87bdf700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a497d68800 0x55a497c13180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c87bdf700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a497d68800 0x55a497c13180 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(5 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0b40 con 0x55a497d68800 2024-09-13T23:45:17.576 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:6 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69c00 con 0x55a497d68c00 2024-09-13T23:45:17.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c883e0700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a497d69c00 0x55a497e02000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:7 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d69c00 2024-09-13T23:45:17.577 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c88be1700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a497d69c00 0x55a497e02000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c88be1700 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a497d69c00 0x55a497e02000 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.578 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d69c00 2024-09-13T23:45:17.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.570+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:8 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68400 2024-09-13T23:45:17.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d68400 2024-09-13T23:45:17.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68400 2024-09-13T23:45:17.579 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d68400 2024-09-13T23:45:17.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:10 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68800 2024-09-13T23:45:17.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0b40 con 0x55a497d68800 2024-09-13T23:45:17.580 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:11 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68400 2024-09-13T23:45:17.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d68400 2024-09-13T23:45:17.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:12 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68c00 2024-09-13T23:45:17.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.581 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68c00 2024-09-13T23:45:17.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(13 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.582 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:14 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68400 2024-09-13T23:45:17.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(14 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d68400 2024-09-13T23:45:17.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.574+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:15 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d69c00 2024-09-13T23:45:17.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.578+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d69c00 2024-09-13T23:45:17.583 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.578+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:16 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68400 2024-09-13T23:45:17.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.578+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d68400 2024-09-13T23:45:17.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.578+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:17 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d69c00 2024-09-13T23:45:17.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.578+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d69c00 2024-09-13T23:45:17.584 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.578+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:18 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68c00 2024-09-13T23:45:17.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.578+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.585 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.578+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:19 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68800 2024-09-13T23:45:17.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.578+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0b40 con 0x55a497d68800 2024-09-13T23:45:17.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.578+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:20 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68400 2024-09-13T23:45:17.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.578+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d68400 2024-09-13T23:45:17.586 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.578+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:21 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68c00 2024-09-13T23:45:17.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.587 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:22 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68800 2024-09-13T23:45:17.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0b40 con 0x55a497d68800 2024-09-13T23:45:17.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:23 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68400 2024-09-13T23:45:17.588 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d68400 2024-09-13T23:45:17.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:24 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d69c00 2024-09-13T23:45:17.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d69c00 2024-09-13T23:45:17.589 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:25 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68400 2024-09-13T23:45:17.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d68400 2024-09-13T23:45:17.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:26 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68800 2024-09-13T23:45:17.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0b40 con 0x55a497d68800 2024-09-13T23:45:17.590 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:27 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d69c00 2024-09-13T23:45:17.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.582+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d69c00 2024-09-13T23:45:17.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:28 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68400 2024-09-13T23:45:17.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d68400 2024-09-13T23:45:17.591 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:29 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d69c00 2024-09-13T23:45:17.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d69c00 2024-09-13T23:45:17.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:30 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68800 2024-09-13T23:45:17.592 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0b40 con 0x55a497d68800 2024-09-13T23:45:17.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:31 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d69c00 2024-09-13T23:45:17.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d69c00 2024-09-13T23:45:17.593 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:32 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68c00 2024-09-13T23:45:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(32 [pgnls start_epoch 35 out=107b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.594 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68c00 2024-09-13T23:45:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(33 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.586+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:34 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68800 2024-09-13T23:45:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.590+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0b40 con 0x55a497d68800 2024-09-13T23:45:17.595 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.590+0000 7f4c87bdf700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:35 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68c00 2024-09-13T23:45:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.590+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(35 [pgnls start_epoch 35 out=115b] v26'2 uv2 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.596 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.590+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497d69800 con 0x55a497d68c00 2024-09-13T23:45:17.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.590+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(36 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.597 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.590+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497e0c000 con 0x55a497d68c00 2024-09-13T23:45:17.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.590+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(37 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.598 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.590+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:38 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497e0c400 con 0x55a497d68c00 2024-09-13T23:45:17.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.590+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(38 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [call out=48b,read 0~874 out=874b] v0'0 uv2 ondisk = 0) v8 ==== 232+0+922 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.599 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.590+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [create,call version.set in=58b,writefull 0~1036 in=1036b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497e0c800 con 0x55a497d68c00 2024-09-13T23:45:17.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.594+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(39 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [create,call,writefull 0~1036] v35'3 uv3 ondisk = 0) v8 ==== 274+0+0 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.602 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.594+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.f 5:f43b6ece:::zone_names.default:head [create,call version.set in=58b,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497e0cc00 con 0x55a497d68c00 2024-09-13T23:45:17.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.598+0000 7f4c883e0700 1 -- 172.21.15.167:0/248759112 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(40 zone_names.default [create,call,writefull 0~46] v35'3 uv3 ondisk = 0) v8 ==== 246+0+0 (crc 0 0 0) 0x55a497dd0900 con 0x55a497d68c00 2024-09-13T23:45:17.605 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.598+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.19 5:9f02474b:::default.zone.:head [create,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55a497e0d000 con 0x55a497d69c00 2024-09-13T23:45:17.608 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 default.zone. [create,writefull 0~46] v35'1 uv1 ondisk = 0) v8 ==== 199+0+0 (crc 0 0 0) 0x55a497dd0240 con 0x55a497d69c00 2024-09-13T23:45:17.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 -- 172.21.15.167:0/135519409 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497cd1000 msgr2=0x55a497c11600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/135519409 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497cd1000 0x55a497c11600 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x55a497c5de90 tx=0x55a497a2f860 comp rx=0 tx=0).stop 2024-09-13T23:45:17.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 -- 172.21.15.167:0/135519409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 msgr2=0x55a497c10b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/135519409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 0x55a497c10b00 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x55a497ceec90 tx=0x55a497c8ad80 comp rx=0 tx=0).stop 2024-09-13T23:45:17.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 -- 172.21.15.167:0/135519409 shutdown_connections 2024-09-13T23:45:17.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/135519409 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497cd1000 0x55a497c11600 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/135519409 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b800 0x55a497c10b00 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.610 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 -- 172.21.15.167:0/135519409 >> 172.21.15.167:0/135519409 conn(0x55a49792b400 msgr2=0x55a497a2b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 -- 172.21.15.167:0/135519409 shutdown_connections 2024-09-13T23:45:17.611 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 -- 172.21.15.167:0/135519409 wait complete. 2024-09-13T23:45:17.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1164112658 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497d69400 msgr2=0x55a497c12c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.612 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/1164112658 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497d69400 0x55a497c12c00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x55a497d7c900 tx=0x55a497a2f8c0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1164112658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 msgr2=0x55a497c11b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/1164112658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 0x55a497c11b80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x55a497d05260 tx=0x55a497a2ff80 comp rx=0 tx=0).stop 2024-09-13T23:45:17.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1164112658 shutdown_connections 2024-09-13T23:45:17.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/1164112658 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497d69400 0x55a497c12c00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/1164112658 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a497cd1800 0x55a497c11b80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1164112658 >> 172.21.15.167:0/1164112658 conn(0x55a497cd1400 msgr2=0x55a497a2c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1164112658 shutdown_connections 2024-09-13T23:45:17.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.602+0000 7f4c89c27f40 1 -- 172.21.15.167:0/1164112658 wait complete. 2024-09-13T23:45:17.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a497d68c00 msgr2=0x55a497c12680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55a497d68c00 0x55a497c12680 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a497d68400 msgr2=0x55a497c12100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.613 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55a497d68400 0x55a497c12100 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a497d69c00 msgr2=0x55a497e02000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55a497d69c00 0x55a497e02000 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a497d68800 msgr2=0x55a497c13180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55a497d68800 0x55a497c13180 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497cd0000 msgr2=0x55a497c11080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55a497cd0000 0x55a497c11080 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x55a497c0b920 tx=0x55a497a2f800 comp rx=0 tx=0).stop 2024-09-13T23:45:17.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c88be1700 1 -- 172.21.15.167:0/248759112 reap_dead start 2024-09-13T23:45:17.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 msgr2=0x55a497c10580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 0x55a497c10580 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x55a4986f7e60 tx=0x55a497a2fa40 comp rx=0 tx=0).stop 2024-09-13T23:45:17.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 shutdown_connections 2024-09-13T23:45:17.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 --2- 172.21.15.167:0/248759112 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55a49792b000 0x55a497c10580 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.614 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 >> 172.21.15.167:0/248759112 conn(0x55a49792ac00 msgr2=0x55a497a2b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 shutdown_connections 2024-09-13T23:45:17.615 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.606+0000 7f4c89c27f40 1 -- 172.21.15.167:0/248759112 wait complete. 2024-09-13T23:45:17.615 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:45:17.615 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "9144208c-3c85-4fb6-9007-2693d5a930f0", 2024-09-13T23:45:17.615 INFO:teuthology.orchestra.run.smithi167.stdout: "name": "default", 2024-09-13T23:45:17.615 INFO:teuthology.orchestra.run.smithi167.stdout: "domain_root": "default.rgw.meta:root", 2024-09-13T23:45:17.615 INFO:teuthology.orchestra.run.smithi167.stdout: "control_pool": "default.rgw.control", 2024-09-13T23:45:17.615 INFO:teuthology.orchestra.run.smithi167.stdout: "gc_pool": "default.rgw.log:gc", 2024-09-13T23:45:17.615 INFO:teuthology.orchestra.run.smithi167.stdout: "lc_pool": "default.rgw.log:lc", 2024-09-13T23:45:17.615 INFO:teuthology.orchestra.run.smithi167.stdout: "log_pool": "default.rgw.log", 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "roles_pool": "default.rgw.meta:roles", 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "otp_pool": "default.rgw.otp", 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "system_key": { 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "", 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "" 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_pools": [ 2024-09-13T23:45:17.616 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "default-placement", 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: "val": { 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": "default.rgw.buckets.index", 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: "storage_classes": { 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: "STANDARD": { 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": "default.rgw.buckets.data" 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: "index_type": 0, 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: "inline_data": true 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:17.617 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "new-placement", 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: "val": { 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": ".rgw.buckets.index.2", 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: "storage_classes": { 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: "STANDARD": { 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": ".rgw.buckets.2" 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: "index_type": 0, 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: "inline_data": false 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:45:17.618 INFO:teuthology.orchestra.run.smithi167.stdout: "realm_id": "", 2024-09-13T23:45:17.619 INFO:teuthology.orchestra.run.smithi167.stdout: "notif_pool": "default.rgw.log:notif" 2024-09-13T23:45:17.619 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:45:17.619 DEBUG:tasks.util.rgw: json result: {'id': '9144208c-3c85-4fb6-9007-2693d5a930f0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-09-13T23:45:17.620 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get'] 2024-09-13T23:45:17.620 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get'] 2024-09-13T23:45:17.620 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get 2024-09-13T23:45:17.661 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:45:17.661 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:45:17.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.654+0000 7fb893642f40 1 Processor -- start 2024-09-13T23:45:17.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.654+0000 7fb893642f40 1 -- start start 2024-09-13T23:45:17.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.654+0000 7fb893642f40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c8a18e1400 0x55c8a1bc6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.662 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.654+0000 7fb893642f40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55c8a19a5500 con 0x55c8a18e1000 2024-09-13T23:45:17.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.654+0000 7fb893642f40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55c8a196ea00 con 0x55c8a18e1400 2024-09-13T23:45:17.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.654+0000 7fb8915fa700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c8a18e1400 0x55c8a1bc6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.654+0000 7fb891dfb700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55c8a18e1000 0x55c8a19e0800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:38808/0 (socket says 172.21.15.167:38808) 2024-09-13T23:45:17.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.654+0000 7fb8915fa700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55c8a18e1400 0x55c8a1bc6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43350/0 (socket says 172.21.15.167:43350) 2024-09-13T23:45:17.663 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.654+0000 7fb891dfb700 1 -- 172.21.15.167:0/710306496 learned_addr learned my addr 172.21.15.167:0/710306496 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:45:17.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb8915fa700 1 -- 172.21.15.167:0/710306496 >> v1:172.21.15.12:6789/0 conn(0x55c8a18e1000 legacy=0x55c8a19e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb8915fa700 1 -- 172.21.15.167:0/710306496 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a18f3d40 con 0x55c8a18e1400 2024-09-13T23:45:17.664 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb8915fa700 1 --2- 172.21.15.167:0/710306496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1400 0x55c8a1bc6580 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55c8a1bc1920 tx=0x55c8a19e5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8747cb4ed851a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb890df9700 1 -- 172.21.15.167:0/710306496 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a19854a0 con 0x55c8a18e1400 2024-09-13T23:45:17.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb890df9700 1 -- 172.21.15.167:0/710306496 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8a19a5800 con 0x55c8a18e1400 2024-09-13T23:45:17.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb890df9700 1 -- 172.21.15.167:0/710306496 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a1985600 con 0x55c8a18e1400 2024-09-13T23:45:17.665 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb893642f40 1 -- 172.21.15.167:0/710306496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1400 msgr2=0x55c8a1bc6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb893642f40 1 --2- 172.21.15.167:0/710306496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1400 0x55c8a1bc6580 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x55c8a1bc1920 tx=0x55c8a19e5920 comp rx=0 tx=0).stop 2024-09-13T23:45:17.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb893642f40 1 -- 172.21.15.167:0/710306496 shutdown_connections 2024-09-13T23:45:17.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb893642f40 1 --2- 172.21.15.167:0/710306496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1400 0x55c8a1bc6580 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb893642f40 1 -- 172.21.15.167:0/710306496 >> 172.21.15.167:0/710306496 conn(0x55c8a18e0c00 msgr2=0x55c8a19e0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb893642f40 1 -- 172.21.15.167:0/710306496 shutdown_connections 2024-09-13T23:45:17.666 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb893642f40 1 -- 172.21.15.167:0/710306496 wait complete. 2024-09-13T23:45:17.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb893642f40 1 Processor -- start 2024-09-13T23:45:17.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb893642f40 1 -- start start 2024-09-13T23:45:17.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb893642f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 0x55c8a1bc6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb893642f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a196e500 con 0x55c8a18e1000 2024-09-13T23:45:17.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb891dfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 0x55c8a1bc6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb891dfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 0x55c8a1bc6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43362/0 (socket says 172.21.15.167:43362) 2024-09-13T23:45:17.667 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.658+0000 7fb891dfb700 1 -- 172.21.15.167:0/4223793634 learned_addr learned my addr 172.21.15.167:0/4223793634 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb891dfb700 1 -- 172.21.15.167:0/4223793634 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a18f3ba0 con 0x55c8a18e1000 2024-09-13T23:45:17.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb891dfb700 1 --2- 172.21.15.167:0/4223793634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 0x55c8a1bc6580 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x55c8a274f8c0 tx=0x55c8a19e5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5219913bd0b70a28 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb8905f8700 1 -- 172.21.15.167:0/4223793634 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a19851e0 con 0x55c8a18e1000 2024-09-13T23:45:17.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb8905f8700 1 -- 172.21.15.167:0/4223793634 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8a19a5b00 con 0x55c8a18e1000 2024-09-13T23:45:17.668 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb8905f8700 1 -- 172.21.15.167:0/4223793634 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a19854a0 con 0x55c8a18e1000 2024-09-13T23:45:17.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb893642f40 1 -- 172.21.15.167:0/4223793634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 msgr2=0x55c8a1bc6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb893642f40 1 --2- 172.21.15.167:0/4223793634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 0x55c8a1bc6580 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x55c8a274f8c0 tx=0x55c8a19e5b60 comp rx=0 tx=0).stop 2024-09-13T23:45:17.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb893642f40 1 -- 172.21.15.167:0/4223793634 shutdown_connections 2024-09-13T23:45:17.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb893642f40 1 --2- 172.21.15.167:0/4223793634 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 0x55c8a1bc6580 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb893642f40 1 -- 172.21.15.167:0/4223793634 >> 172.21.15.167:0/4223793634 conn(0x55c8a18e0c00 msgr2=0x55c8a19e1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb893642f40 1 -- 172.21.15.167:0/4223793634 shutdown_connections 2024-09-13T23:45:17.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb893642f40 1 -- 172.21.15.167:0/4223793634 wait complete. 2024-09-13T23:45:17.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb893642f40 1 Processor -- start 2024-09-13T23:45:17.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb893642f40 1 -- start start 2024-09-13T23:45:17.670 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb893642f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 0x55c8a1bc6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb893642f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a196e3c0 con 0x55c8a18e1000 2024-09-13T23:45:17.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb891dfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 0x55c8a1bc6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb891dfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 0x55c8a1bc6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43372/0 (socket says 172.21.15.167:43372) 2024-09-13T23:45:17.671 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb891dfb700 1 -- 172.21.15.167:0/3086115012 learned_addr learned my addr 172.21.15.167:0/3086115012 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb891dfb700 1 -- 172.21.15.167:0/3086115012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a18f3a00 con 0x55c8a18e1000 2024-09-13T23:45:17.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.662+0000 7fb891dfb700 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 0x55c8a1bc6580 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x55c8a274faa0 tx=0x55c8a19e5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb88edf5700 1 -- 172.21.15.167:0/3086115012 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a1985600 con 0x55c8a18e1000 2024-09-13T23:45:17.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c8a18f3860 con 0x55c8a18e1000 2024-09-13T23:45:17.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb88edf5700 1 -- 172.21.15.167:0/3086115012 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8a19a5c80 con 0x55c8a18e1000 2024-09-13T23:45:17.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb88edf5700 1 -- 172.21.15.167:0/3086115012 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a19851e0 con 0x55c8a18e1000 2024-09-13T23:45:17.672 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8a18f36c0 con 0x55c8a18e1000 2024-09-13T23:45:17.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb893642f40 1 Processor -- start 2024-09-13T23:45:17.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb893642f40 1 -- start start 2024-09-13T23:45:17.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb893642f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 0x55c8a1bc6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb893642f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a196e000 con 0x55c8a18e1800 2024-09-13T23:45:17.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb88edf5700 1 -- 172.21.15.167:0/3086115012 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c8a19dab00 con 0x55c8a18e1000 2024-09-13T23:45:17.673 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb8925fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 0x55c8a1bc6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb8925fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 0x55c8a1bc6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43378/0 (socket says 172.21.15.167:43378) 2024-09-13T23:45:17.674 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb8925fc700 1 -- 172.21.15.167:0/1751946085 learned_addr learned my addr 172.21.15.167:0/1751946085 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb88edf5700 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1c9a000 0x55c8a1bc7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb88edf5700 1 -- 172.21.15.167:0/3086115012 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55c8a1947180 con 0x55c8a18e1000 2024-09-13T23:45:17.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb891dfb700 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1c9a000 0x55c8a1bc7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb8925fc700 1 -- 172.21.15.167:0/1751946085 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a18f2000 con 0x55c8a18e1800 2024-09-13T23:45:17.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb8925fc700 1 --2- 172.21.15.167:0/1751946085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 0x55c8a1bc6b00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55c8a1c78060 tx=0x55c8a1c4ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f34ee9d9b8f2ff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb891dfb700 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1c9a000 0x55c8a1bc7080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55c8a1c01650 tx=0x55c8a1c4e240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb88c5f0700 1 -- 172.21.15.167:0/1751946085 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a1985600 con 0x55c8a18e1800 2024-09-13T23:45:17.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb88c5f0700 1 -- 172.21.15.167:0/1751946085 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8a1cb0000 con 0x55c8a18e1800 2024-09-13T23:45:17.675 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.666+0000 7fb88c5f0700 1 -- 172.21.15.167:0/1751946085 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a19851e0 con 0x55c8a18e1800 2024-09-13T23:45:17.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb893642f40 1 -- 172.21.15.167:0/1751946085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 msgr2=0x55c8a1bc6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb893642f40 1 --2- 172.21.15.167:0/1751946085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 0x55c8a1bc6b00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x55c8a1c78060 tx=0x55c8a1c4ede0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb893642f40 1 -- 172.21.15.167:0/1751946085 shutdown_connections 2024-09-13T23:45:17.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb893642f40 1 --2- 172.21.15.167:0/1751946085 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 0x55c8a1bc6b00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb893642f40 1 -- 172.21.15.167:0/1751946085 >> 172.21.15.167:0/1751946085 conn(0x55c8a18e1400 msgr2=0x55c8a19e1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb893642f40 1 -- 172.21.15.167:0/1751946085 shutdown_connections 2024-09-13T23:45:17.676 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb893642f40 1 -- 172.21.15.167:0/1751946085 wait complete. 2024-09-13T23:45:17.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb893642f40 1 Processor -- start 2024-09-13T23:45:17.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb893642f40 1 -- start start 2024-09-13T23:45:17.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb893642f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 0x55c8a1bc6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.677 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb893642f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a1906640 con 0x55c8a18e1800 2024-09-13T23:45:17.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb8925fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 0x55c8a1bc6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb8925fc700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 0x55c8a1bc6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43382/0 (socket says 172.21.15.167:43382) 2024-09-13T23:45:17.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb8925fc700 1 -- 172.21.15.167:0/1054672474 learned_addr learned my addr 172.21.15.167:0/1054672474 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.678 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb8925fc700 1 -- 172.21.15.167:0/1054672474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a18f2ea0 con 0x55c8a18e1800 2024-09-13T23:45:17.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.670+0000 7fb8925fc700 1 --2- 172.21.15.167:0/1054672474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 0x55c8a1bc6b00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x55c8a1c00420 tx=0x55c8a1c4e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb88aded700 1 -- 172.21.15.167:0/1054672474 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a1985a20 con 0x55c8a18e1800 2024-09-13T23:45:17.679 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb88aded700 1 -- 172.21.15.167:0/1054672474 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8a1cb0180 con 0x55c8a18e1800 2024-09-13T23:45:17.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb893642f40 1 -- 172.21.15.167:0/1054672474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c8a18f2820 con 0x55c8a18e1800 2024-09-13T23:45:17.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb88aded700 1 -- 172.21.15.167:0/1054672474 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a1985600 con 0x55c8a18e1800 2024-09-13T23:45:17.680 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb893642f40 1 -- 172.21.15.167:0/1054672474 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8a18f29c0 con 0x55c8a18e1800 2024-09-13T23:45:17.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb88aded700 1 -- 172.21.15.167:0/1054672474 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c8a19dadc0 con 0x55c8a18e1800 2024-09-13T23:45:17.681 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb893642f40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:45:17.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb893642f40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:45:17.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb893642f40 1 Processor -- start 2024-09-13T23:45:17.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb893642f40 1 -- start start 2024-09-13T23:45:17.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb88aded700 1 --2- 172.21.15.167:0/1054672474 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1c9bc00 0x55c8a1bc7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb88aded700 1 -- 172.21.15.167:0/1054672474 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55c8a1947340 con 0x55c8a18e1800 2024-09-13T23:45:17.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb893642f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 0x55c8a1bc7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb893642f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a19068c0 con 0x55c8a1c9b000 2024-09-13T23:45:17.682 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb8925fc700 1 --2- 172.21.15.167:0/1054672474 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1c9bc00 0x55c8a1bc7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb891dfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 0x55c8a1bc7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb891dfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 0x55c8a1bc7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43388/0 (socket says 172.21.15.167:43388) 2024-09-13T23:45:17.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb891dfb700 1 -- 172.21.15.167:0/2036949014 learned_addr learned my addr 172.21.15.167:0/2036949014 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.683 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb8925fc700 1 --2- 172.21.15.167:0/1054672474 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1c9bc00 0x55c8a1bc7600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55c8a1c79ef0 tx=0x55c8a1c4f7a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb891dfb700 1 -- 172.21.15.167:0/2036949014 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a18f2b60 con 0x55c8a1c9b000 2024-09-13T23:45:17.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.674+0000 7fb891dfb700 1 --2- 172.21.15.167:0/2036949014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 0x55c8a1bc7b80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x55c8a1c008d0 tx=0x55c8a1c4f0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fa8da724eb09c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb878dc9700 1 -- 172.21.15.167:0/2036949014 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a19854a0 con 0x55c8a1c9b000 2024-09-13T23:45:17.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb878dc9700 1 -- 172.21.15.167:0/2036949014 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8a1cb0600 con 0x55c8a1c9b000 2024-09-13T23:45:17.684 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb878dc9700 1 -- 172.21.15.167:0/2036949014 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a19858c0 con 0x55c8a1c9b000 2024-09-13T23:45:17.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 -- 172.21.15.167:0/2036949014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 msgr2=0x55c8a1bc7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 --2- 172.21.15.167:0/2036949014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 0x55c8a1bc7b80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x55c8a1c008d0 tx=0x55c8a1c4f0e0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 -- 172.21.15.167:0/2036949014 shutdown_connections 2024-09-13T23:45:17.685 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 --2- 172.21.15.167:0/2036949014 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 0x55c8a1bc7b80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 -- 172.21.15.167:0/2036949014 >> 172.21.15.167:0/2036949014 conn(0x55c8a1c9ac00 msgr2=0x55c8a19e2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 -- 172.21.15.167:0/2036949014 shutdown_connections 2024-09-13T23:45:17.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 -- 172.21.15.167:0/2036949014 wait complete. 2024-09-13T23:45:17.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 Processor -- start 2024-09-13T23:45:17.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 -- start start 2024-09-13T23:45:17.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 0x55c8a1bc7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8a1906f00 con 0x55c8a1c9b000 2024-09-13T23:45:17.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb891dfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 0x55c8a1bc7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb891dfb700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 0x55c8a1bc7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43400/0 (socket says 172.21.15.167:43400) 2024-09-13T23:45:17.686 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb891dfb700 1 -- 172.21.15.167:0/3616889493 learned_addr learned my addr 172.21.15.167:0/3616889493 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb891dfb700 1 -- 172.21.15.167:0/3616889493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8a18f2d00 con 0x55c8a1c9b000 2024-09-13T23:45:17.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb891dfb700 1 --2- 172.21.15.167:0/3616889493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 0x55c8a1bc7b80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x55c8a1c79380 tx=0x55c8a1c4e1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb8775c6700 1 -- 172.21.15.167:0/3616889493 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a19854a0 con 0x55c8a1c9b000 2024-09-13T23:45:17.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb8775c6700 1 -- 172.21.15.167:0/3616889493 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8a1cb0300 con 0x55c8a1c9b000 2024-09-13T23:45:17.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb8775c6700 1 -- 172.21.15.167:0/3616889493 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55c8a19858c0 con 0x55c8a1c9b000 2024-09-13T23:45:17.687 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 -- 172.21.15.167:0/3616889493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c8a18f2340 con 0x55c8a1c9b000 2024-09-13T23:45:17.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.678+0000 7fb893642f40 1 -- 172.21.15.167:0/3616889493 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8a1d58000 con 0x55c8a1c9b000 2024-09-13T23:45:17.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb8775c6700 1 -- 172.21.15.167:0/3616889493 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55c8a19dbb80 con 0x55c8a1c9b000 2024-09-13T23:45:17.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb8775c6700 1 --2- 172.21.15.167:0/3616889493 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1d51c00 0x55c8a1bc8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.688 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb8775c6700 1 -- 172.21.15.167:0/3616889493 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55c8a1947500 con 0x55c8a1c9b000 2024-09-13T23:45:17.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb8915fa700 1 --2- 172.21.15.167:0/3616889493 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1d51c00 0x55c8a1bc8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb8915fa700 1 --2- 172.21.15.167:0/3616889493 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1d51c00 0x55c8a1bc8c00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55c8a1bc1920 tx=0x55c8a19e5800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb8755c2700 20 reqs_thread_entry: start 2024-09-13T23:45:17.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb893642f40 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c8a1d50800 0x55c8a1bc8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.689 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c8a1d50c00 con 0x55c8a1d50800 2024-09-13T23:45:17.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb8925fc700 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c8a1d50800 0x55c8a1bc8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.690 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb8925fc700 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c8a1d50800 0x55c8a1bc8100 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb8925fc700 1 -- 172.21.15.167:0/3086115012 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55c8a1db8240 con 0x55c8a1d50800 2024-09-13T23:45:17.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb893642f40 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c8a1d51000 0x55c8a1bc8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.682+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c8a1d51400 con 0x55c8a1d51000 2024-09-13T23:45:17.691 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.686+0000 7fb891dfb700 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c8a1d51000 0x55c8a1bc8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.686+0000 7fb891dfb700 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c8a1d51000 0x55c8a1bc8680 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.692 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.686+0000 7fb891dfb700 1 -- 172.21.15.167:0/3086115012 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55c8a1db8900 con 0x55c8a1d51000 2024-09-13T23:45:17.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.686+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:3 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55c8a1d51800 con 0x55c8a1d51000 2024-09-13T23:45:17.693 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.686+0000 7fb891dfb700 1 -- 172.21.15.167:0/3086115012 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(3 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [call out=48b,read 0~1036 out=1036b] v0'0 uv3 ondisk = 0) v8 ==== 232+0+1084 (crc 0 0 0) 0x55c8a1db8900 con 0x55c8a1d51000 2024-09-13T23:45:17.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/1054672474 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1c9bc00 msgr2=0x55c8a1bc7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/1054672474 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1c9bc00 0x55c8a1bc7600 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55c8a1c79ef0 tx=0x55c8a1c4f7a0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/1054672474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 msgr2=0x55c8a1bc6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.695 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/1054672474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 0x55c8a1bc6b00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x55c8a1c00420 tx=0x55c8a1c4e840 comp rx=0 tx=0).stop 2024-09-13T23:45:17.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/1054672474 shutdown_connections 2024-09-13T23:45:17.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/1054672474 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1c9bc00 0x55c8a1bc7600 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/1054672474 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1800 0x55c8a1bc6b00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/1054672474 >> 172.21.15.167:0/1054672474 conn(0x55c8a18e1400 msgr2=0x55c8a19e1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/1054672474 shutdown_connections 2024-09-13T23:45:17.696 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/1054672474 wait complete. 2024-09-13T23:45:17.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3616889493 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1d51c00 msgr2=0x55c8a1bc8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.697 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/3616889493 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1d51c00 0x55c8a1bc8c00 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x55c8a1bc1920 tx=0x55c8a19e5800 comp rx=0 tx=0).stop 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3616889493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 msgr2=0x55c8a1bc7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/3616889493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 0x55c8a1bc7b80 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x55c8a1c79380 tx=0x55c8a1c4e1e0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3616889493 shutdown_connections 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/3616889493 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1d51c00 0x55c8a1bc8c00 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/3616889493 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a1c9b000 0x55c8a1bc7b80 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3616889493 >> 172.21.15.167:0/3616889493 conn(0x55c8a1c9ac00 msgr2=0x55c8a19e2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3616889493 shutdown_connections 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3616889493 wait complete. 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c8a1d51000 msgr2=0x55c8a1bc8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c8a1d51000 0x55c8a1bc8680 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c8a1d50800 msgr2=0x55c8a1bc8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c8a1d50800 0x55c8a1bc8100 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.698 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1c9a000 msgr2=0x55c8a1bc7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1c9a000 0x55c8a1bc7080 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55c8a1c01650 tx=0x55c8a1c4e240 comp rx=0 tx=0).stop 2024-09-13T23:45:17.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 msgr2=0x55c8a1bc6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 0x55c8a1bc6580 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x55c8a274faa0 tx=0x55c8a19e5a40 comp rx=0 tx=0).stop 2024-09-13T23:45:17.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 shutdown_connections 2024-09-13T23:45:17.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55c8a1d51000 0x55c8a1bc8680 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55c8a1d50800 0x55c8a1bc8100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55c8a1c9a000 0x55c8a1bc7080 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 --2- 172.21.15.167:0/3086115012 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55c8a18e1000 0x55c8a1bc6580 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 >> 172.21.15.167:0/3086115012 conn(0x55c8a18e0c00 msgr2=0x55c8a19e1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 shutdown_connections 2024-09-13T23:45:17.699 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.690+0000 7fb893642f40 1 -- 172.21.15.167:0/3086115012 wait complete. 2024-09-13T23:45:17.700 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:45:17.700 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "9144208c-3c85-4fb6-9007-2693d5a930f0", 2024-09-13T23:45:17.700 INFO:teuthology.orchestra.run.smithi167.stdout: "name": "default", 2024-09-13T23:45:17.700 INFO:teuthology.orchestra.run.smithi167.stdout: "domain_root": "default.rgw.meta:root", 2024-09-13T23:45:17.700 INFO:teuthology.orchestra.run.smithi167.stdout: "control_pool": "default.rgw.control", 2024-09-13T23:45:17.700 INFO:teuthology.orchestra.run.smithi167.stdout: "gc_pool": "default.rgw.log:gc", 2024-09-13T23:45:17.700 INFO:teuthology.orchestra.run.smithi167.stdout: "lc_pool": "default.rgw.log:lc", 2024-09-13T23:45:17.700 INFO:teuthology.orchestra.run.smithi167.stdout: "log_pool": "default.rgw.log", 2024-09-13T23:45:17.700 INFO:teuthology.orchestra.run.smithi167.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-09-13T23:45:17.700 INFO:teuthology.orchestra.run.smithi167.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-09-13T23:45:17.700 INFO:teuthology.orchestra.run.smithi167.stdout: "roles_pool": "default.rgw.meta:roles", 2024-09-13T23:45:17.701 INFO:teuthology.orchestra.run.smithi167.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-09-13T23:45:17.701 INFO:teuthology.orchestra.run.smithi167.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-09-13T23:45:17.701 INFO:teuthology.orchestra.run.smithi167.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-09-13T23:45:17.701 INFO:teuthology.orchestra.run.smithi167.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-09-13T23:45:17.701 INFO:teuthology.orchestra.run.smithi167.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-09-13T23:45:17.701 INFO:teuthology.orchestra.run.smithi167.stdout: "otp_pool": "default.rgw.otp", 2024-09-13T23:45:17.701 INFO:teuthology.orchestra.run.smithi167.stdout: "system_key": { 2024-09-13T23:45:17.701 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "", 2024-09-13T23:45:17.701 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "" 2024-09-13T23:45:17.701 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.701 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_pools": [ 2024-09-13T23:45:17.701 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "default-placement", 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: "val": { 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": "default.rgw.buckets.index", 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: "storage_classes": { 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: "STANDARD": { 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": "default.rgw.buckets.data" 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: "index_type": 0, 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: "inline_data": true 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.702 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "new-placement", 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: "val": { 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": ".rgw.buckets.index.2", 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: "storage_classes": { 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: "STANDARD": { 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": ".rgw.buckets.2" 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: "index_type": 0, 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: "inline_data": false 2024-09-13T23:45:17.703 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.704 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.704 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:45:17.704 INFO:teuthology.orchestra.run.smithi167.stdout: "realm_id": "", 2024-09-13T23:45:17.704 INFO:teuthology.orchestra.run.smithi167.stdout: "notif_pool": "default.rgw.log:notif" 2024-09-13T23:45:17.704 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:45:17.705 DEBUG:tasks.util.rgw: json result: {'id': '9144208c-3c85-4fb6-9007-2693d5a930f0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-09-13T23:45:17.705 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-09-13T23:45:17.705 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-09-13T23:45:17.705 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-09-13T23:45:17.789 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:45:17.789 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:45:17.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.782+0000 7fbec678cf40 1 Processor -- start 2024-09-13T23:45:17.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.782+0000 7fbec678cf40 1 -- start start 2024-09-13T23:45:17.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.782+0000 7fbec678cf40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x555988551000 0x555988836580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.782+0000 7fbec678cf40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x555988615500 con 0x555988551400 2024-09-13T23:45:17.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.782+0000 7fbec678cf40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x5559885dea00 con 0x555988551000 2024-09-13T23:45:17.790 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.782+0000 7fbec4f45700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x555988551000 0x555988836580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.782+0000 7fbec4744700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x555988551400 0x555988650800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:38820/0 (socket says 172.21.15.167:38820) 2024-09-13T23:45:17.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.782+0000 7fbec4744700 1 -- 172.21.15.167:0/3970945208 learned_addr learned my addr 172.21.15.167:0/3970945208 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:45:17.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec4f45700 1 -- 172.21.15.167:0/3970945208 >> v1:172.21.15.12:6789/0 conn(0x555988551400 legacy=0x555988650800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec4f45700 1 -- 172.21.15.167:0/3970945208 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555988563d40 con 0x555988551000 2024-09-13T23:45:17.791 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec4f45700 1 --2- 172.21.15.167:0/3970945208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x555988831a70 tx=0x555988655920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e90de0a5781ec51 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec3f43700 1 -- 172.21.15.167:0/3970945208 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f54a0 con 0x555988551000 2024-09-13T23:45:17.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec3f43700 1 -- 172.21.15.167:0/3970945208 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555988615800 con 0x555988551000 2024-09-13T23:45:17.792 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec3f43700 1 -- 172.21.15.167:0/3970945208 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f5600 con 0x555988551000 2024-09-13T23:45:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec678cf40 1 -- 172.21.15.167:0/3970945208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 msgr2=0x555988836580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec678cf40 1 --2- 172.21.15.167:0/3970945208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x555988831a70 tx=0x555988655920 comp rx=0 tx=0).stop 2024-09-13T23:45:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec678cf40 1 -- 172.21.15.167:0/3970945208 shutdown_connections 2024-09-13T23:45:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec678cf40 1 --2- 172.21.15.167:0/3970945208 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec678cf40 1 -- 172.21.15.167:0/3970945208 >> 172.21.15.167:0/3970945208 conn(0x555988550c00 msgr2=0x555988650000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec678cf40 1 -- 172.21.15.167:0/3970945208 shutdown_connections 2024-09-13T23:45:17.793 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec678cf40 1 -- 172.21.15.167:0/3970945208 wait complete. 2024-09-13T23:45:17.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec678cf40 1 Processor -- start 2024-09-13T23:45:17.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec678cf40 1 -- start start 2024-09-13T23:45:17.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec678cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.794 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec678cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559885de500 con 0x555988551000 2024-09-13T23:45:17.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec4f45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec4f45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43424/0 (socket says 172.21.15.167:43424) 2024-09-13T23:45:17.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.786+0000 7fbec4f45700 1 -- 172.21.15.167:0/2764331039 learned_addr learned my addr 172.21.15.167:0/2764331039 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.795 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec4f45700 1 -- 172.21.15.167:0/2764331039 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555988563ba0 con 0x555988551000 2024-09-13T23:45:17.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec4f45700 1 --2- 172.21.15.167:0/2764331039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x5559886ac0c0 tx=0x555988655800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d93f61fe3cc37f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.796 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec3742700 1 -- 172.21.15.167:0/2764331039 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f5760 con 0x555988551000 2024-09-13T23:45:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec3742700 1 -- 172.21.15.167:0/2764331039 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555988615b00 con 0x555988551000 2024-09-13T23:45:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec3742700 1 -- 172.21.15.167:0/2764331039 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f51e0 con 0x555988551000 2024-09-13T23:45:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec678cf40 1 -- 172.21.15.167:0/2764331039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 msgr2=0x555988836580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec678cf40 1 --2- 172.21.15.167:0/2764331039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x5559886ac0c0 tx=0x555988655800 comp rx=0 tx=0).stop 2024-09-13T23:45:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec678cf40 1 -- 172.21.15.167:0/2764331039 shutdown_connections 2024-09-13T23:45:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec678cf40 1 --2- 172.21.15.167:0/2764331039 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec678cf40 1 -- 172.21.15.167:0/2764331039 >> 172.21.15.167:0/2764331039 conn(0x555988550c00 msgr2=0x555988651000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec678cf40 1 -- 172.21.15.167:0/2764331039 shutdown_connections 2024-09-13T23:45:17.797 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec678cf40 1 -- 172.21.15.167:0/2764331039 wait complete. 2024-09-13T23:45:17.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec678cf40 1 Processor -- start 2024-09-13T23:45:17.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec678cf40 1 -- start start 2024-09-13T23:45:17.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec678cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec678cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559885de3c0 con 0x555988551000 2024-09-13T23:45:17.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec4f45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.798 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec4f45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43428/0 (socket says 172.21.15.167:43428) 2024-09-13T23:45:17.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec4f45700 1 -- 172.21.15.167:0/3911153436 learned_addr learned my addr 172.21.15.167:0/3911153436 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec4f45700 1 -- 172.21.15.167:0/3911153436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555988563a00 con 0x555988551000 2024-09-13T23:45:17.799 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.790+0000 7fbec4f45700 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x5559886adf20 tx=0x555988655920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec1f3f700 1 -- 172.21.15.167:0/3911153436 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f54a0 con 0x555988551000 2024-09-13T23:45:17.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec1f3f700 1 -- 172.21.15.167:0/3911153436 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555988615c80 con 0x555988551000 2024-09-13T23:45:17.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 -- 172.21.15.167:0/3911153436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555988563860 con 0x555988551000 2024-09-13T23:45:17.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec1f3f700 1 -- 172.21.15.167:0/3911153436 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f5600 con 0x555988551000 2024-09-13T23:45:17.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 -- 172.21.15.167:0/3911153436 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559885636c0 con 0x555988551000 2024-09-13T23:45:17.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 Processor -- start 2024-09-13T23:45:17.800 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 -- start start 2024-09-13T23:45:17.801 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 0x555988836b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559885de000 con 0x555988551800 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec5746700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 0x555988836b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec5746700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 0x555988836b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43432/0 (socket says 172.21.15.167:43432) 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec5746700 1 -- 172.21.15.167:0/3975706766 learned_addr learned my addr 172.21.15.167:0/3975706766 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec1f3f700 1 -- 172.21.15.167:0/3911153436 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55598864ab00 con 0x555988551000 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec5746700 1 -- 172.21.15.167:0/3975706766 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555988562000 con 0x555988551800 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec5746700 1 --2- 172.21.15.167:0/3975706766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 0x555988836b00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x555988884d20 tx=0x5559888c28a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df315786c433b7b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec1f3f700 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555988904000 0x555988837080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec1f3f700 1 -- 172.21.15.167:0/3911153436 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5559885b7180 con 0x555988551000 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec4f45700 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555988904000 0x555988837080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbebf73a700 1 -- 172.21.15.167:0/3975706766 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f54a0 con 0x555988551800 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbebf73a700 1 -- 172.21.15.167:0/3975706766 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55598890a000 con 0x555988551800 2024-09-13T23:45:17.802 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbebf73a700 1 -- 172.21.15.167:0/3975706766 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f5600 con 0x555988551800 2024-09-13T23:45:17.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 -- 172.21.15.167:0/3975706766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 msgr2=0x555988836b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 --2- 172.21.15.167:0/3975706766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 0x555988836b00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x555988884d20 tx=0x5559888c28a0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec4f45700 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555988904000 0x555988837080 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x555988906030 tx=0x5559888c2120 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 -- 172.21.15.167:0/3975706766 shutdown_connections 2024-09-13T23:45:17.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 --2- 172.21.15.167:0/3975706766 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 0x555988836b00 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 -- 172.21.15.167:0/3975706766 >> 172.21.15.167:0/3975706766 conn(0x555988551400 msgr2=0x555988651800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 -- 172.21.15.167:0/3975706766 shutdown_connections 2024-09-13T23:45:17.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 -- 172.21.15.167:0/3975706766 wait complete. 2024-09-13T23:45:17.804 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 Processor -- start 2024-09-13T23:45:17.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.794+0000 7fbec678cf40 1 -- start start 2024-09-13T23:45:17.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbec678cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 0x555988836b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbec678cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555988576640 con 0x555988551800 2024-09-13T23:45:17.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbec5746700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 0x555988836b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbec5746700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 0x555988836b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43448/0 (socket says 172.21.15.167:43448) 2024-09-13T23:45:17.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbec5746700 1 -- 172.21.15.167:0/1164631373 learned_addr learned my addr 172.21.15.167:0/1164631373 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.805 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbec5746700 1 -- 172.21.15.167:0/1164631373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555988562ea0 con 0x555988551800 2024-09-13T23:45:17.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbec5746700 1 --2- 172.21.15.167:0/1164631373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 0x555988836b00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x5559889064e0 tx=0x555988655b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbebdf37700 1 -- 172.21.15.167:0/1164631373 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f5760 con 0x555988551800 2024-09-13T23:45:17.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbebdf37700 1 -- 172.21.15.167:0/1164631373 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55598890a180 con 0x555988551800 2024-09-13T23:45:17.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbebdf37700 1 -- 172.21.15.167:0/1164631373 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f54a0 con 0x555988551800 2024-09-13T23:45:17.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbec678cf40 1 -- 172.21.15.167:0/1164631373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555988562820 con 0x555988551800 2024-09-13T23:45:17.806 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbec678cf40 1 -- 172.21.15.167:0/1164631373 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559885629c0 con 0x555988551800 2024-09-13T23:45:17.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbebdf37700 1 -- 172.21.15.167:0/1164631373 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55598864b080 con 0x555988551800 2024-09-13T23:45:17.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbebdf37700 1 --2- 172.21.15.167:0/1164631373 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555988904c00 0x555988837600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbebdf37700 1 -- 172.21.15.167:0/1164631373 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5559885b7340 con 0x555988551800 2024-09-13T23:45:17.807 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.798+0000 7fbec4744700 1 --2- 172.21.15.167:0/1164631373 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555988904c00 0x555988837600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:45:17.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:45:17.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec4744700 1 --2- 172.21.15.167:0/1164631373 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555988904c00 0x555988837600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x555988907b60 tx=0x5559888c3f80 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 1 Processor -- start 2024-09-13T23:45:17.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 1 -- start start 2024-09-13T23:45:17.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 0x555988837b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.808 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559885768c0 con 0x555988905400 2024-09-13T23:45:17.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec4f45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 0x555988837b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec4f45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 0x555988837b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43452/0 (socket says 172.21.15.167:43452) 2024-09-13T23:45:17.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec4f45700 1 -- 172.21.15.167:0/597120069 learned_addr learned my addr 172.21.15.167:0/597120069 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.809 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec4f45700 1 -- 172.21.15.167:0/597120069 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555988562b60 con 0x555988905400 2024-09-13T23:45:17.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec4f45700 1 --2- 172.21.15.167:0/597120069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 0x555988837b80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x5559889061b0 tx=0x5559888c21e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d3ead9e272d9c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbeabf13700 1 -- 172.21.15.167:0/597120069 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f5760 con 0x555988905400 2024-09-13T23:45:17.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbeabf13700 1 -- 172.21.15.167:0/597120069 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55598890a600 con 0x555988905400 2024-09-13T23:45:17.810 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbeabf13700 1 -- 172.21.15.167:0/597120069 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f54a0 con 0x555988905400 2024-09-13T23:45:17.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 1 -- 172.21.15.167:0/597120069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 msgr2=0x555988837b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 1 --2- 172.21.15.167:0/597120069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 0x555988837b80 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x5559889061b0 tx=0x5559888c21e0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 1 -- 172.21.15.167:0/597120069 shutdown_connections 2024-09-13T23:45:17.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 1 --2- 172.21.15.167:0/597120069 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 0x555988837b80 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 1 -- 172.21.15.167:0/597120069 >> 172.21.15.167:0/597120069 conn(0x555988905000 msgr2=0x555988652000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 1 -- 172.21.15.167:0/597120069 shutdown_connections 2024-09-13T23:45:17.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 1 -- 172.21.15.167:0/597120069 wait complete. 2024-09-13T23:45:17.811 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.802+0000 7fbec678cf40 1 Processor -- start 2024-09-13T23:45:17.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbec678cf40 1 -- start start 2024-09-13T23:45:17.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbec678cf40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 0x555988837b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbec678cf40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x555988576f00 con 0x555988905400 2024-09-13T23:45:17.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbec4f45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 0x555988837b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.812 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbec4f45700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 0x555988837b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43464/0 (socket says 172.21.15.167:43464) 2024-09-13T23:45:17.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbec4f45700 1 -- 172.21.15.167:0/1319961194 learned_addr learned my addr 172.21.15.167:0/1319961194 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbec4f45700 1 -- 172.21.15.167:0/1319961194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555988562d00 con 0x555988905400 2024-09-13T23:45:17.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbec4f45700 1 --2- 172.21.15.167:0/1319961194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 0x555988837b80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x555988959140 tx=0x5559888c20c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.813 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbeaa710700 1 -- 172.21.15.167:0/1319961194 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f58c0 con 0x555988905400 2024-09-13T23:45:17.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbeaa710700 1 -- 172.21.15.167:0/1319961194 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55598890a300 con 0x555988905400 2024-09-13T23:45:17.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbeaa710700 1 -- 172.21.15.167:0/1319961194 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x5559885f5a20 con 0x555988905400 2024-09-13T23:45:17.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbec678cf40 1 -- 172.21.15.167:0/1319961194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555988562340 con 0x555988905400 2024-09-13T23:45:17.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbec678cf40 1 -- 172.21.15.167:0/1319961194 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559889c4000 con 0x555988905400 2024-09-13T23:45:17.814 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbeaa710700 1 -- 172.21.15.167:0/1319961194 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55598864adc0 con 0x555988905400 2024-09-13T23:45:17.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbea870c700 20 reqs_thread_entry: start 2024-09-13T23:45:17.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbec678cf40 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5559889be800 0x555988838100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbec678cf40 1 -- 172.21.15.167:0/3911153436 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5559889bec00 con 0x5559889be800 2024-09-13T23:45:17.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.806+0000 7fbeaa710700 1 --2- 172.21.15.167:0/1319961194 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5559889bfc00 0x555988838c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.810+0000 7fbeaa710700 1 -- 172.21.15.167:0/1319961194 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x5559885b7500 con 0x555988905400 2024-09-13T23:45:17.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.810+0000 7fbec5746700 1 --2- 172.21.15.167:0/1319961194 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5559889bfc00 0x555988838c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.815 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.810+0000 7fbec4744700 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5559889be800 0x555988838100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.810+0000 7fbec4744700 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5559889be800 0x555988838100 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.810+0000 7fbec5746700 1 --2- 172.21.15.167:0/1319961194 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5559889bfc00 0x555988838c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x5559889e25a0 tx=0x5559888c33e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.816 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.810+0000 7fbec4744700 1 -- 172.21.15.167:0/3911153436 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x555988a38240 con 0x5559889be800 2024-09-13T23:45:17.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.810+0000 7fbec678cf40 1 -- 172.21.15.167:0/3911153436 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:2 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x5559889bf000 con 0x5559889be800 2024-09-13T23:45:17.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.810+0000 7fbec4744700 1 -- 172.21.15.167:0/3911153436 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(2 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [call out=48b,read 0~1036 out=1036b] v0'0 uv3 ondisk = 0) v8 ==== 232+0+1084 (crc 0 0 0) 0x555988a38240 con 0x5559889be800 2024-09-13T23:45:17.817 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.810+0000 7fbec678cf40 1 -- 172.21.15.167:0/3911153436 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:3 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~874 in=874b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x5559889bf400 con 0x5559889be800 2024-09-13T23:45:17.821 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.814+0000 7fbec4744700 1 -- 172.21.15.167:0/3911153436 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(3 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [stat,call,call,writefull 0~874] v35'4 uv4 ondisk = 0) v8 ==== 316+0+0 (crc 0 0 0) 0x555988a38240 con 0x5559889be800 2024-09-13T23:45:17.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.814+0000 7fbec678cf40 1 -- 172.21.15.167:0/1164631373 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555988904c00 msgr2=0x555988837600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.814+0000 7fbec678cf40 1 --2- 172.21.15.167:0/1164631373 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555988904c00 0x555988837600 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x555988907b60 tx=0x5559888c3f80 comp rx=0 tx=0).stop 2024-09-13T23:45:17.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.814+0000 7fbec678cf40 1 -- 172.21.15.167:0/1164631373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 msgr2=0x555988836b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.814+0000 7fbec678cf40 1 --2- 172.21.15.167:0/1164631373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 0x555988836b00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x5559889064e0 tx=0x555988655b60 comp rx=0 tx=0).stop 2024-09-13T23:45:17.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.814+0000 7fbec678cf40 1 -- 172.21.15.167:0/1164631373 shutdown_connections 2024-09-13T23:45:17.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.814+0000 7fbec678cf40 1 --2- 172.21.15.167:0/1164631373 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555988904c00 0x555988837600 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.814+0000 7fbec678cf40 1 --2- 172.21.15.167:0/1164631373 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551800 0x555988836b00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.814+0000 7fbec678cf40 1 -- 172.21.15.167:0/1164631373 >> 172.21.15.167:0/1164631373 conn(0x555988551400 msgr2=0x555988651800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.814+0000 7fbec678cf40 1 -- 172.21.15.167:0/1164631373 shutdown_connections 2024-09-13T23:45:17.823 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.814+0000 7fbec678cf40 1 -- 172.21.15.167:0/1164631373 wait complete. 2024-09-13T23:45:17.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/1319961194 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5559889bfc00 msgr2=0x555988838c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 --2- 172.21.15.167:0/1319961194 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5559889bfc00 0x555988838c00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x5559889e25a0 tx=0x5559888c33e0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/1319961194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 msgr2=0x555988837b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 --2- 172.21.15.167:0/1319961194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 0x555988837b80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x555988959140 tx=0x5559888c20c0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/1319961194 shutdown_connections 2024-09-13T23:45:17.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 --2- 172.21.15.167:0/1319961194 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x5559889bfc00 0x555988838c00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.825 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 --2- 172.21.15.167:0/1319961194 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988905400 0x555988837b80 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/1319961194 >> 172.21.15.167:0/1319961194 conn(0x555988905000 msgr2=0x555988652000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/1319961194 shutdown_connections 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/1319961194 wait complete. 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/3911153436 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5559889be800 msgr2=0x555988838100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5559889be800 0x555988838100 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/3911153436 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555988904000 msgr2=0x555988837080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555988904000 0x555988837080 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x555988906030 tx=0x5559888c2120 comp rx=0 tx=0).stop 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/3911153436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 msgr2=0x555988836580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x5559886adf20 tx=0x555988655920 comp rx=0 tx=0).stop 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/3911153436 shutdown_connections 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x5559889be800 0x555988838100 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x555988904000 0x555988837080 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 --2- 172.21.15.167:0/3911153436 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x555988551000 0x555988836580 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.826 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/3911153436 >> 172.21.15.167:0/3911153436 conn(0x555988550c00 msgr2=0x555988651000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/3911153436 shutdown_connections 2024-09-13T23:45:17.827 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.818+0000 7fbec678cf40 1 -- 172.21.15.167:0/3911153436 wait complete. 2024-09-13T23:45:17.827 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:45:17.827 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "9144208c-3c85-4fb6-9007-2693d5a930f0", 2024-09-13T23:45:17.827 INFO:teuthology.orchestra.run.smithi167.stdout: "name": "default", 2024-09-13T23:45:17.827 INFO:teuthology.orchestra.run.smithi167.stdout: "domain_root": "default.rgw.meta:root", 2024-09-13T23:45:17.827 INFO:teuthology.orchestra.run.smithi167.stdout: "control_pool": "default.rgw.control", 2024-09-13T23:45:17.827 INFO:teuthology.orchestra.run.smithi167.stdout: "gc_pool": "default.rgw.log:gc", 2024-09-13T23:45:17.827 INFO:teuthology.orchestra.run.smithi167.stdout: "lc_pool": "default.rgw.log:lc", 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "log_pool": "default.rgw.log", 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "intent_log_pool": "default.rgw.log:intent", 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "usage_log_pool": "default.rgw.log:usage", 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "roles_pool": "default.rgw.meta:roles", 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "reshard_pool": "default.rgw.log:reshard", 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "user_keys_pool": "default.rgw.meta:users.keys", 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "user_email_pool": "default.rgw.meta:users.email", 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "user_swift_pool": "default.rgw.meta:users.swift", 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "user_uid_pool": "default.rgw.meta:users.uid", 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "otp_pool": "default.rgw.otp", 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "system_key": { 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "", 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "" 2024-09-13T23:45:17.828 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_pools": [ 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: "key": "default-placement", 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: "val": { 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: "index_pool": "default.rgw.buckets.index", 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: "storage_classes": { 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: "STANDARD": { 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: "data_pool": "default.rgw.buckets.data" 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: "data_extra_pool": "default.rgw.buckets.non-ec", 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: "index_type": 0, 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: "inline_data": true 2024-09-13T23:45:17.829 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.830 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.830 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:45:17.830 INFO:teuthology.orchestra.run.smithi167.stdout: "realm_id": "", 2024-09-13T23:45:17.830 INFO:teuthology.orchestra.run.smithi167.stdout: "notif_pool": "default.rgw.log:notif" 2024-09-13T23:45:17.830 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:45:17.831 DEBUG:tasks.util.rgw: json result: {'id': '9144208c-3c85-4fb6-9007-2693d5a930f0', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': '', 'notif_pool': 'default.rgw.log:notif'} 2024-09-13T23:45:17.831 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'get'] 2024-09-13T23:45:17.831 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'get'] 2024-09-13T23:45:17.831 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup get 2024-09-13T23:45:17.867 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:45:17.867 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:45:17.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f980ff40 1 Processor -- start 2024-09-13T23:45:17.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f980ff40 1 -- start start 2024-09-13T23:45:17.868 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f980ff40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x558a0f75d000 0x558a0fa42580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f980ff40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x558a0f821500 con 0x558a0f75d400 2024-09-13T23:45:17.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f980ff40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x558a0f7eaa00 con 0x558a0f75d000 2024-09-13T23:45:17.869 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f7fc8700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x558a0f75d000 0x558a0fa42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f77c7700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x558a0f75d400 0x558a0f85c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:38834/0 (socket says 172.21.15.167:38834) 2024-09-13T23:45:17.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f77c7700 1 -- 172.21.15.167:0/2530097081 learned_addr learned my addr 172.21.15.167:0/2530097081 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:45:17.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f7fc8700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x558a0f75d000 0x558a0fa42580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43472/0 (socket says 172.21.15.167:43472) 2024-09-13T23:45:17.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f7fc8700 1 -- 172.21.15.167:0/2530097081 >> v1:172.21.15.12:6789/0 conn(0x558a0f75d400 legacy=0x558a0f85c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f6fc6700 1 -- 172.21.15.167:0/2530097081 <== mon.0 v1:172.21.15.12:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219844181 0 0) 0x558a0f8b8000 con 0x558a0f75d400 2024-09-13T23:45:17.870 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f7fc8700 1 -- 172.21.15.167:0/2530097081 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a0f76fd40 con 0x558a0f75d000 2024-09-13T23:45:17.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f7fc8700 1 --2- 172.21.15.167:0/2530097081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x558a0fa3da70 tx=0x558a0f861920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5644b2ab974ac729 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f6fc6700 1 -- 172.21.15.167:0/2530097081 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f8014a0 con 0x558a0f75d000 2024-09-13T23:45:17.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f6fc6700 1 -- 172.21.15.167:0/2530097081 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558a0f821800 con 0x558a0f75d000 2024-09-13T23:45:17.871 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.862+0000 7f52f6fc6700 1 -- 172.21.15.167:0/2530097081 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f801600 con 0x558a0f75d000 2024-09-13T23:45:17.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f980ff40 1 -- 172.21.15.167:0/2530097081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 msgr2=0x558a0fa42580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2530097081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x558a0fa3da70 tx=0x558a0f861920 comp rx=0 tx=0).stop 2024-09-13T23:45:17.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f980ff40 1 -- 172.21.15.167:0/2530097081 shutdown_connections 2024-09-13T23:45:17.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2530097081 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f980ff40 1 -- 172.21.15.167:0/2530097081 >> 172.21.15.167:0/2530097081 conn(0x558a0f75cc00 msgr2=0x558a0f85c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f980ff40 1 -- 172.21.15.167:0/2530097081 shutdown_connections 2024-09-13T23:45:17.872 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f980ff40 1 -- 172.21.15.167:0/2530097081 wait complete. 2024-09-13T23:45:17.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f980ff40 1 Processor -- start 2024-09-13T23:45:17.873 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f980ff40 1 -- start start 2024-09-13T23:45:17.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f980ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f980ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0f7ea500 con 0x558a0f75d000 2024-09-13T23:45:17.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f7fc8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f7fc8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43482/0 (socket says 172.21.15.167:43482) 2024-09-13T23:45:17.874 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f7fc8700 1 -- 172.21.15.167:0/1111676822 learned_addr learned my addr 172.21.15.167:0/1111676822 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.866+0000 7f52f7fc8700 1 -- 172.21.15.167:0/1111676822 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a0f76fba0 con 0x558a0f75d000 2024-09-13T23:45:17.875 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f7fc8700 1 --2- 172.21.15.167:0/1111676822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x558a10527e00 tx=0x558a0f861800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c7fc4963eafc43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f67c5700 1 -- 172.21.15.167:0/1111676822 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f801760 con 0x558a0f75d000 2024-09-13T23:45:17.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f67c5700 1 -- 172.21.15.167:0/1111676822 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558a0f821b00 con 0x558a0f75d000 2024-09-13T23:45:17.876 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f67c5700 1 -- 172.21.15.167:0/1111676822 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f8014a0 con 0x558a0f75d000 2024-09-13T23:45:17.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f980ff40 1 -- 172.21.15.167:0/1111676822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 msgr2=0x558a0fa42580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f980ff40 1 --2- 172.21.15.167:0/1111676822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x558a10527e00 tx=0x558a0f861800 comp rx=0 tx=0).stop 2024-09-13T23:45:17.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f980ff40 1 -- 172.21.15.167:0/1111676822 shutdown_connections 2024-09-13T23:45:17.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f980ff40 1 --2- 172.21.15.167:0/1111676822 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f980ff40 1 -- 172.21.15.167:0/1111676822 >> 172.21.15.167:0/1111676822 conn(0x558a0f75cc00 msgr2=0x558a0f85d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f980ff40 1 -- 172.21.15.167:0/1111676822 shutdown_connections 2024-09-13T23:45:17.877 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f980ff40 1 -- 172.21.15.167:0/1111676822 wait complete. 2024-09-13T23:45:17.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f980ff40 1 Processor -- start 2024-09-13T23:45:17.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f980ff40 1 -- start start 2024-09-13T23:45:17.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f980ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.878 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f980ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0f7ea3c0 con 0x558a0f75d000 2024-09-13T23:45:17.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f7fc8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f7fc8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43498/0 (socket says 172.21.15.167:43498) 2024-09-13T23:45:17.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.870+0000 7f52f7fc8700 1 -- 172.21.15.167:0/2141566558 learned_addr learned my addr 172.21.15.167:0/2141566558 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.879 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f7fc8700 1 -- 172.21.15.167:0/2141566558 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a0f76fa00 con 0x558a0f75d000 2024-09-13T23:45:17.880 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f7fc8700 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x558a10527f20 tx=0x558a0f861920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f4fc2700 1 -- 172.21.15.167:0/2141566558 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f801600 con 0x558a0f75d000 2024-09-13T23:45:17.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f4fc2700 1 -- 172.21.15.167:0/2141566558 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558a0f821c80 con 0x558a0f75d000 2024-09-13T23:45:17.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f4fc2700 1 -- 172.21.15.167:0/2141566558 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f8018c0 con 0x558a0f75d000 2024-09-13T23:45:17.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558a0f76f860 con 0x558a0f75d000 2024-09-13T23:45:17.881 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a0f76f6c0 con 0x558a0f75d000 2024-09-13T23:45:17.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f980ff40 1 Processor -- start 2024-09-13T23:45:17.882 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f980ff40 1 -- start start 2024-09-13T23:45:17.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f980ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 0x558a0fa42b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f4fc2700 1 -- 172.21.15.167:0/2141566558 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x558a0f856b00 con 0x558a0f75d000 2024-09-13T23:45:17.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f980ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0f7ea000 con 0x558a0f75d800 2024-09-13T23:45:17.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f4fc2700 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0fafe000 0x558a0fa43080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f87c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 0x558a0fa42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f4fc2700 1 -- 172.21.15.167:0/2141566558 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x558a0f7c3180 con 0x558a0f75d000 2024-09-13T23:45:17.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f87c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 0x558a0fa42b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43510/0 (socket says 172.21.15.167:43510) 2024-09-13T23:45:17.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f87c9700 1 -- 172.21.15.167:0/1606403177 learned_addr learned my addr 172.21.15.167:0/1606403177 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.883 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.874+0000 7f52f7fc8700 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0fafe000 0x558a0fa43080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f87c9700 1 -- 172.21.15.167:0/1606403177 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a0f76e000 con 0x558a0f75d800 2024-09-13T23:45:17.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f87c9700 1 --2- 172.21.15.167:0/1606403177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 0x558a0fa42b00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x558a10527fb0 tx=0x558a0fac0b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f22dae6d625f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.884 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f7fc8700 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0fafe000 0x558a0fa43080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x558a105d1560 tx=0x558a0fac0c60 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f27bd700 1 -- 172.21.15.167:0/1606403177 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f801600 con 0x558a0f75d800 2024-09-13T23:45:17.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f27bd700 1 -- 172.21.15.167:0/1606403177 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558a0fb1e000 con 0x558a0f75d800 2024-09-13T23:45:17.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f27bd700 1 -- 172.21.15.167:0/1606403177 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f8018c0 con 0x558a0f75d800 2024-09-13T23:45:17.885 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f980ff40 1 -- 172.21.15.167:0/1606403177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 msgr2=0x558a0fa42b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f980ff40 1 --2- 172.21.15.167:0/1606403177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 0x558a0fa42b00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x558a10527fb0 tx=0x558a0fac0b40 comp rx=0 tx=0).stop 2024-09-13T23:45:17.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f980ff40 1 -- 172.21.15.167:0/1606403177 shutdown_connections 2024-09-13T23:45:17.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f980ff40 1 --2- 172.21.15.167:0/1606403177 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 0x558a0fa42b00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f980ff40 1 -- 172.21.15.167:0/1606403177 >> 172.21.15.167:0/1606403177 conn(0x558a0f75d400 msgr2=0x558a0f85d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f980ff40 1 -- 172.21.15.167:0/1606403177 shutdown_connections 2024-09-13T23:45:17.886 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f980ff40 1 -- 172.21.15.167:0/1606403177 wait complete. 2024-09-13T23:45:17.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f980ff40 1 Processor -- start 2024-09-13T23:45:17.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f980ff40 1 -- start start 2024-09-13T23:45:17.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f980ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 0x558a0fa42b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.878+0000 7f52f980ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0f782640 con 0x558a0f75d800 2024-09-13T23:45:17.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f87c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 0x558a0fa42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.887 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f87c9700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 0x558a0fa42b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43516/0 (socket says 172.21.15.167:43516) 2024-09-13T23:45:17.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f87c9700 1 -- 172.21.15.167:0/1428996894 learned_addr learned my addr 172.21.15.167:0/1428996894 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f87c9700 1 -- 172.21.15.167:0/1428996894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a0f76eea0 con 0x558a0f75d800 2024-09-13T23:45:17.888 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f87c9700 1 --2- 172.21.15.167:0/1428996894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 0x558a0fa42b00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x558a105d03f0 tx=0x558a0f8618c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f0fba700 1 -- 172.21.15.167:0/1428996894 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f801760 con 0x558a0f75d800 2024-09-13T23:45:17.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f0fba700 1 -- 172.21.15.167:0/1428996894 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558a0fb1e180 con 0x558a0f75d800 2024-09-13T23:45:17.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f980ff40 1 -- 172.21.15.167:0/1428996894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558a0f76e820 con 0x558a0f75d800 2024-09-13T23:45:17.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f0fba700 1 -- 172.21.15.167:0/1428996894 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f801600 con 0x558a0f75d800 2024-09-13T23:45:17.889 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f980ff40 1 -- 172.21.15.167:0/1428996894 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a0f76e9c0 con 0x558a0f75d800 2024-09-13T23:45:17.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f980ff40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:45:17.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f980ff40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:45:17.890 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f0fba700 1 -- 172.21.15.167:0/1428996894 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x558a0f856dc0 con 0x558a0f75d800 2024-09-13T23:45:17.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f980ff40 1 Processor -- start 2024-09-13T23:45:17.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.882+0000 7f52f980ff40 1 -- start start 2024-09-13T23:45:17.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f0fba700 1 --2- 172.21.15.167:0/1428996894 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0faffc00 0x558a0fa43600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 0x558a0fa43b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.891 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f0fba700 1 -- 172.21.15.167:0/1428996894 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x558a0f7c3340 con 0x558a0f75d800 2024-09-13T23:45:17.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0f7828c0 con 0x558a0faff000 2024-09-13T23:45:17.892 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f87c9700 1 --2- 172.21.15.167:0/1428996894 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0faffc00 0x558a0fa43600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f7fc8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 0x558a0fa43b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f7fc8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 0x558a0fa43b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43522/0 (socket says 172.21.15.167:43522) 2024-09-13T23:45:17.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f7fc8700 1 -- 172.21.15.167:0/2041222992 learned_addr learned my addr 172.21.15.167:0/2041222992 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f87c9700 1 --2- 172.21.15.167:0/1428996894 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0faffc00 0x558a0fa43600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x558a0fae2930 tx=0x558a0fac1500 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f7fc8700 1 -- 172.21.15.167:0/2041222992 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a0f76eb60 con 0x558a0faff000 2024-09-13T23:45:17.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f7fc8700 1 --2- 172.21.15.167:0/2041222992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 0x558a0fa43b80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x558a105d0900 tx=0x558a0fac0de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e96d0b764689959 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52def96700 1 -- 172.21.15.167:0/2041222992 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f801760 con 0x558a0faff000 2024-09-13T23:45:17.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52def96700 1 -- 172.21.15.167:0/2041222992 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558a0fb1e600 con 0x558a0faff000 2024-09-13T23:45:17.893 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52def96700 1 -- 172.21.15.167:0/2041222992 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f801600 con 0x558a0faff000 2024-09-13T23:45:17.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 -- 172.21.15.167:0/2041222992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 msgr2=0x558a0fa43b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.894 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2041222992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 0x558a0fa43b80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x558a105d0900 tx=0x558a0fac0de0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 -- 172.21.15.167:0/2041222992 shutdown_connections 2024-09-13T23:45:17.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2041222992 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 0x558a0fa43b80 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 -- 172.21.15.167:0/2041222992 >> 172.21.15.167:0/2041222992 conn(0x558a0fafec00 msgr2=0x558a0f85e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 -- 172.21.15.167:0/2041222992 shutdown_connections 2024-09-13T23:45:17.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 -- 172.21.15.167:0/2041222992 wait complete. 2024-09-13T23:45:17.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 Processor -- start 2024-09-13T23:45:17.895 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 -- start start 2024-09-13T23:45:17.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 0x558a0fa43b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.886+0000 7f52f980ff40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a0f782f00 con 0x558a0faff000 2024-09-13T23:45:17.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52f7fc8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 0x558a0fa43b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52f7fc8700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 0x558a0fa43b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43534/0 (socket says 172.21.15.167:43534) 2024-09-13T23:45:17.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52f7fc8700 1 -- 172.21.15.167:0/2117454332 learned_addr learned my addr 172.21.15.167:0/2117454332 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52f7fc8700 1 -- 172.21.15.167:0/2117454332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a0f76ed00 con 0x558a0faff000 2024-09-13T23:45:17.896 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52f7fc8700 1 --2- 172.21.15.167:0/2117454332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 0x558a0fa43b80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x558a0fae3bf0 tx=0x558a0fac0f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52dd793700 1 -- 172.21.15.167:0/2117454332 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f801a20 con 0x558a0faff000 2024-09-13T23:45:17.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52dd793700 1 -- 172.21.15.167:0/2117454332 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558a0fb1e300 con 0x558a0faff000 2024-09-13T23:45:17.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52dd793700 1 -- 172.21.15.167:0/2117454332 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x558a0f801b80 con 0x558a0faff000 2024-09-13T23:45:17.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52f980ff40 1 -- 172.21.15.167:0/2117454332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558a0f76e340 con 0x558a0faff000 2024-09-13T23:45:17.897 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52f980ff40 1 -- 172.21.15.167:0/2117454332 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a0fbc8000 con 0x558a0faff000 2024-09-13T23:45:17.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52dd793700 1 -- 172.21.15.167:0/2117454332 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x558a0f857b80 con 0x558a0faff000 2024-09-13T23:45:17.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52dd793700 1 --2- 172.21.15.167:0/2117454332 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0fbbdc00 0x558a0fa44c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52dd793700 1 -- 172.21.15.167:0/2117454332 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x558a0f7c3500 con 0x558a0faff000 2024-09-13T23:45:17.898 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.890+0000 7f52f77c7700 1 --2- 172.21.15.167:0/2117454332 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0fbbdc00 0x558a0fa44c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52f77c7700 1 --2- 172.21.15.167:0/2117454332 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0fbbdc00 0x558a0fa44c00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x558a10526030 tx=0x558a0fbc7080 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.899 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52db78f700 20 reqs_thread_entry: start 2024-09-13T23:45:17.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x558a0fbbc800 0x558a0fa44100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x558a0fbbcc00 con 0x558a0fbbc800 2024-09-13T23:45:17.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52f87c9700 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x558a0fbbc800 0x558a0fa44100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.900 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52f87c9700 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x558a0fbbc800 0x558a0fa44100 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52f87c9700 1 -- 172.21.15.167:0/2141566558 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x558a0fc2c240 con 0x558a0fbbc800 2024-09-13T23:45:17.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x558a0fbbd000 0x558a0fa44680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:2 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x558a0fbbd400 con 0x558a0fbbd000 2024-09-13T23:45:17.901 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52f7fc8700 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x558a0fbbd000 0x558a0fa44680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.902 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52f7fc8700 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x558a0fbbd000 0x558a0fa44680 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52f7fc8700 1 -- 172.21.15.167:0/2141566558 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(2 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x558a0fc2c900 con 0x558a0fbbd000 2024-09-13T23:45:17.903 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.894+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:3 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x558a0fbbd800 con 0x558a0fbbd000 2024-09-13T23:45:17.904 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f7fc8700 1 -- 172.21.15.167:0/2141566558 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(3 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [call out=24b,read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 237+0+444 (crc 0 0 0) 0x558a0fc2c900 con 0x558a0fbbd000 2024-09-13T23:45:17.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/1428996894 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0faffc00 msgr2=0x558a0fa43600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/1428996894 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0faffc00 0x558a0fa43600 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x558a0fae2930 tx=0x558a0fac1500 comp rx=0 tx=0).stop 2024-09-13T23:45:17.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/1428996894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 msgr2=0x558a0fa42b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.905 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/1428996894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 0x558a0fa42b00 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x558a105d03f0 tx=0x558a0f8618c0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/1428996894 shutdown_connections 2024-09-13T23:45:17.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/1428996894 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0faffc00 0x558a0fa43600 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/1428996894 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d800 0x558a0fa42b00 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/1428996894 >> 172.21.15.167:0/1428996894 conn(0x558a0f75d400 msgr2=0x558a0f85d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/1428996894 shutdown_connections 2024-09-13T23:45:17.906 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/1428996894 wait complete. 2024-09-13T23:45:17.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/2117454332 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0fbbdc00 msgr2=0x558a0fa44c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2117454332 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0fbbdc00 0x558a0fa44c00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x558a10526030 tx=0x558a0fbc7080 comp rx=0 tx=0).stop 2024-09-13T23:45:17.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/2117454332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 msgr2=0x558a0fa43b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2117454332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 0x558a0fa43b80 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x558a0fae3bf0 tx=0x558a0fac0f00 comp rx=0 tx=0).stop 2024-09-13T23:45:17.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/2117454332 shutdown_connections 2024-09-13T23:45:17.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2117454332 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0fbbdc00 0x558a0fa44c00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2117454332 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0faff000 0x558a0fa43b80 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/2117454332 >> 172.21.15.167:0/2117454332 conn(0x558a0fafec00 msgr2=0x558a0f85e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/2117454332 shutdown_connections 2024-09-13T23:45:17.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/2117454332 wait complete. 2024-09-13T23:45:17.908 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x558a0fbbd000 msgr2=0x558a0fa44680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x558a0fbbd000 0x558a0fa44680 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x558a0fbbc800 msgr2=0x558a0fa44100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x558a0fbbc800 0x558a0fa44100 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0fafe000 msgr2=0x558a0fa43080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0fafe000 0x558a0fa43080 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x558a105d1560 tx=0x558a0fac0c60 comp rx=0 tx=0).stop 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 msgr2=0x558a0fa42580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x558a10527f20 tx=0x558a0f861920 comp rx=0 tx=0).stop 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 shutdown_connections 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x558a0fbbd000 0x558a0fa44680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x558a0fbbc800 0x558a0fa44100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x558a0fafe000 0x558a0fa43080 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 --2- 172.21.15.167:0/2141566558 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x558a0f75d000 0x558a0fa42580 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.909 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.898+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 >> 172.21.15.167:0/2141566558 conn(0x558a0f75cc00 msgr2=0x558a0f85d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.902+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 shutdown_connections 2024-09-13T23:45:17.910 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.902+0000 7f52f980ff40 1 -- 172.21.15.167:0/2141566558 wait complete. 2024-09-13T23:45:17.910 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:45:17.910 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "363ea8c8-5049-424d-855a-cd6979bf7acf", 2024-09-13T23:45:17.910 INFO:teuthology.orchestra.run.smithi167.stdout: "name": "default", 2024-09-13T23:45:17.910 INFO:teuthology.orchestra.run.smithi167.stdout: "api_name": "default", 2024-09-13T23:45:17.910 INFO:teuthology.orchestra.run.smithi167.stdout: "is_master": true, 2024-09-13T23:45:17.910 INFO:teuthology.orchestra.run.smithi167.stdout: "endpoints": [], 2024-09-13T23:45:17.910 INFO:teuthology.orchestra.run.smithi167.stdout: "hostnames": [], 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: "hostnames_s3website": [], 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: "master_zone": "9144208c-3c85-4fb6-9007-2693d5a930f0", 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: "zones": [ 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: "id": "9144208c-3c85-4fb6-9007-2693d5a930f0", 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: "name": "default", 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: "endpoints": [], 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: "log_meta": false, 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: "log_data": false, 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_index_max_shards": 11, 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: "read_only": false, 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: "tier_type": "", 2024-09-13T23:45:17.911 INFO:teuthology.orchestra.run.smithi167.stdout: "sync_from_all": true, 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: "sync_from": [], 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: "redirect_zone": "", 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: "supported_features": [ 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: "compress-encrypted", 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: "resharding" 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_targets": [ 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: "name": "default-placement", 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: "tags": [], 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: "storage_classes": [ 2024-09-13T23:45:17.912 INFO:teuthology.orchestra.run.smithi167.stdout: "STANDARD" 2024-09-13T23:45:17.913 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:45:17.913 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:17.913 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:45:17.913 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "default-placement", 2024-09-13T23:45:17.913 INFO:teuthology.orchestra.run.smithi167.stdout: "realm_id": "", 2024-09-13T23:45:17.913 INFO:teuthology.orchestra.run.smithi167.stdout: "sync_policy": { 2024-09-13T23:45:17.913 INFO:teuthology.orchestra.run.smithi167.stdout: "groups": [] 2024-09-13T23:45:17.913 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:17.913 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled_features": [ 2024-09-13T23:45:17.913 INFO:teuthology.orchestra.run.smithi167.stdout: "resharding" 2024-09-13T23:45:17.913 INFO:teuthology.orchestra.run.smithi167.stdout: ] 2024-09-13T23:45:17.913 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:45:17.914 DEBUG:tasks.util.rgw: json result: {'id': '363ea8c8-5049-424d-855a-cd6979bf7acf', 'name': 'default', 'api_name': 'default', 'is_master': True, 'endpoints': [], 'hostnames': [], 'hostnames_s3website': [], 'master_zone': '9144208c-3c85-4fb6-9007-2693d5a930f0', 'zones': [{'id': '9144208c-3c85-4fb6-9007-2693d5a930f0', 'name': 'default', 'endpoints': [], 'log_meta': False, 'log_data': False, 'bucket_index_max_shards': 11, 'read_only': False, 'tier_type': '', 'sync_from_all': True, 'sync_from': [], 'redirect_zone': '', 'supported_features': ['compress-encrypted', 'resharding']}], 'placement_targets': [{'name': 'default-placement', 'tags': [], 'storage_classes': ['STANDARD']}], 'default_placement': 'default-placement', 'realm_id': '', 'sync_policy': {'groups': []}, 'enabled_features': ['resharding']} 2024-09-13T23:45:17.920 INFO:teuthology.run_tasks:Running task radosgw-admin-rest... 2024-09-13T23:45:17.937 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-09-13T23:45:17.937 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-09-13T23:45:17.937 DEBUG:teuthology.orchestra.run.smithi167:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid ada --display-name 'Ms. Admin User' --access-key MH1WC2XQ1S8UISFDZC8W --secret dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG --max-buckets 0 --caps 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write' 2024-09-13T23:45:17.991 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setuser ceph since I am not root 2024-09-13T23:45:17.991 INFO:teuthology.orchestra.run.smithi167.stderr:ignoring --setgroup ceph since I am not root 2024-09-13T23:45:17.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc922f6df40 1 Processor -- start 2024-09-13T23:45:17.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc922f6df40 1 -- start start 2024-09-13T23:45:17.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc922f6df40 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55989c1a7400 0x55989c48c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc922f6df40 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- 0x55989c26b500 con 0x55989c1a7000 2024-09-13T23:45:17.992 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc922f6df40 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x55989c234a00 con 0x55989c1a7400 2024-09-13T23:45:17.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc920f25700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x55989c1a7400 0x55989c48c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc921726700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x55989c1a7000 0x55989c2a6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.167:38836/0 (socket says 172.21.15.167:38836) 2024-09-13T23:45:17.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc921726700 1 -- 172.21.15.167:0/3912598930 learned_addr learned my addr 172.21.15.167:0/3912598930 (peer_addr_for_me v1:172.21.15.167:0/0) 2024-09-13T23:45:17.993 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc920f25700 1 -- 172.21.15.167:0/3912598930 >> v1:172.21.15.12:6789/0 conn(0x55989c1a7000 legacy=0x55989c2a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc920f25700 1 -- 172.21.15.167:0/3912598930 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55989c1b9d40 con 0x55989c1a7400 2024-09-13T23:45:17.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc920f25700 1 --2- 172.21.15.167:0/3912598930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7400 0x55989c48c580 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x55989c487920 tx=0x55989c2ab920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1c9b8d0ddfc583 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc920724700 1 -- 172.21.15.167:0/3912598930 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c24b4a0 con 0x55989c1a7400 2024-09-13T23:45:17.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc920724700 1 -- 172.21.15.167:0/3912598930 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55989c26b800 con 0x55989c1a7400 2024-09-13T23:45:17.994 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc920724700 1 -- 172.21.15.167:0/3912598930 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c24b600 con 0x55989c1a7400 2024-09-13T23:45:17.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc922f6df40 1 -- 172.21.15.167:0/3912598930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7400 msgr2=0x55989c48c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc922f6df40 1 --2- 172.21.15.167:0/3912598930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7400 0x55989c48c580 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x55989c487920 tx=0x55989c2ab920 comp rx=0 tx=0).stop 2024-09-13T23:45:17.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc922f6df40 1 -- 172.21.15.167:0/3912598930 shutdown_connections 2024-09-13T23:45:17.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc922f6df40 1 --2- 172.21.15.167:0/3912598930 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7400 0x55989c48c580 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:17.995 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc922f6df40 1 -- 172.21.15.167:0/3912598930 >> 172.21.15.167:0/3912598930 conn(0x55989c1a6c00 msgr2=0x55989c2a6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:17.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.986+0000 7fc922f6df40 1 -- 172.21.15.167:0/3912598930 shutdown_connections 2024-09-13T23:45:17.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 -- 172.21.15.167:0/3912598930 wait complete. 2024-09-13T23:45:17.996 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 Processor -- start 2024-09-13T23:45:17.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 -- start start 2024-09-13T23:45:17.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 0x55989c48c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:17.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55989c234500 con 0x55989c1a7000 2024-09-13T23:45:17.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc921726700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 0x55989c48c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:17.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc921726700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 0x55989c48c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43556/0 (socket says 172.21.15.167:43556) 2024-09-13T23:45:17.997 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc921726700 1 -- 172.21.15.167:0/3937099893 learned_addr learned my addr 172.21.15.167:0/3937099893 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:17.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc921726700 1 -- 172.21.15.167:0/3937099893 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55989c1b9ba0 con 0x55989c1a7000 2024-09-13T23:45:17.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc921726700 1 --2- 172.21.15.167:0/3937099893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 0x55989c48c580 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x55989d015ec0 tx=0x55989c2abb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7812a0b83ddc96d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:17.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc91ff23700 1 -- 172.21.15.167:0/3937099893 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c24b1e0 con 0x55989c1a7000 2024-09-13T23:45:17.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc91ff23700 1 -- 172.21.15.167:0/3937099893 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55989c26bb00 con 0x55989c1a7000 2024-09-13T23:45:17.998 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc91ff23700 1 -- 172.21.15.167:0/3937099893 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c24b4a0 con 0x55989c1a7000 2024-09-13T23:45:17.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 -- 172.21.15.167:0/3937099893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 msgr2=0x55989c48c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:17.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 --2- 172.21.15.167:0/3937099893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 0x55989c48c580 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x55989d015ec0 tx=0x55989c2abb60 comp rx=0 tx=0).stop 2024-09-13T23:45:17.999 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 -- 172.21.15.167:0/3937099893 shutdown_connections 2024-09-13T23:45:18.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 --2- 172.21.15.167:0/3937099893 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 0x55989c48c580 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 -- 172.21.15.167:0/3937099893 >> 172.21.15.167:0/3937099893 conn(0x55989c1a6c00 msgr2=0x55989c2a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:18.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 -- 172.21.15.167:0/3937099893 shutdown_connections 2024-09-13T23:45:18.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 -- 172.21.15.167:0/3937099893 wait complete. 2024-09-13T23:45:18.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 Processor -- start 2024-09-13T23:45:18.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.990+0000 7fc922f6df40 1 -- start start 2024-09-13T23:45:18.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc922f6df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 0x55989c48c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:18.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc922f6df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55989c2343c0 con 0x55989c1a7000 2024-09-13T23:45:18.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc921726700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 0x55989c48c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:18.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc921726700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 0x55989c48c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43558/0 (socket says 172.21.15.167:43558) 2024-09-13T23:45:18.000 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc921726700 1 -- 172.21.15.167:0/3453674734 learned_addr learned my addr 172.21.15.167:0/3453674734 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:18.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc921726700 1 -- 172.21.15.167:0/3453674734 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55989c1b9a00 con 0x55989c1a7000 2024-09-13T23:45:18.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc921726700 1 --2- 172.21.15.167:0/3453674734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 0x55989c48c580 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x55989d015f80 tx=0x55989c2aba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:18.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc91e720700 1 -- 172.21.15.167:0/3453674734 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c24b600 con 0x55989c1a7000 2024-09-13T23:45:18.001 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc922f6df40 1 -- 172.21.15.167:0/3453674734 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55989c1b9860 con 0x55989c1a7000 2024-09-13T23:45:18.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc91e720700 1 -- 172.21.15.167:0/3453674734 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55989c26bc80 con 0x55989c1a7000 2024-09-13T23:45:18.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc91e720700 1 -- 172.21.15.167:0/3453674734 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c24b1e0 con 0x55989c1a7000 2024-09-13T23:45:18.002 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc922f6df40 1 -- 172.21.15.167:0/3453674734 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55989c1b96c0 con 0x55989c1a7000 2024-09-13T23:45:18.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc922f6df40 1 Processor -- start 2024-09-13T23:45:18.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc922f6df40 1 -- start start 2024-09-13T23:45:18.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc922f6df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 0x55989c48cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:18.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc922f6df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55989c1cc640 con 0x55989c1a7800 2024-09-13T23:45:18.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc91e720700 1 -- 172.21.15.167:0/3453674734 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55989c2a0b00 con 0x55989c1a7000 2024-09-13T23:45:18.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc91e720700 1 --2- 172.21.15.167:0/3453674734 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c558000 0x55989c48d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:18.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc91e720700 1 -- 172.21.15.167:0/3453674734 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55989c20d180 con 0x55989c1a7000 2024-09-13T23:45:18.003 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.994+0000 7fc921f27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 0x55989c48cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:18.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc921f27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 0x55989c48cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43568/0 (socket says 172.21.15.167:43568) 2024-09-13T23:45:18.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc921f27700 1 -- 172.21.15.167:0/4024598496 learned_addr learned my addr 172.21.15.167:0/4024598496 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:18.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc921726700 1 --2- 172.21.15.167:0/3453674734 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c558000 0x55989c48d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:18.004 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc921f27700 1 -- 172.21.15.167:0/4024598496 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55989c1b8000 con 0x55989c1a7800 2024-09-13T23:45:18.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc921726700 1 --2- 172.21.15.167:0/3453674734 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c558000 0x55989c48d080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55989c4c7110 tx=0x55989c510240 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:18.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc921f27700 1 --2- 172.21.15.167:0/4024598496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 0x55989c48cb00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55989c4c63f0 tx=0x55989c5110e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2650d40a702558 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:18.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc91bf1b700 1 -- 172.21.15.167:0/4024598496 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c24b600 con 0x55989c1a7800 2024-09-13T23:45:18.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc91bf1b700 1 -- 172.21.15.167:0/4024598496 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55989c56a000 con 0x55989c1a7800 2024-09-13T23:45:18.005 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc91bf1b700 1 -- 172.21.15.167:0/4024598496 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c24b1e0 con 0x55989c1a7800 2024-09-13T23:45:18.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc922f6df40 1 -- 172.21.15.167:0/4024598496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 msgr2=0x55989c48cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:18.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc922f6df40 1 --2- 172.21.15.167:0/4024598496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 0x55989c48cb00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55989c4c63f0 tx=0x55989c5110e0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc922f6df40 1 -- 172.21.15.167:0/4024598496 shutdown_connections 2024-09-13T23:45:18.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc922f6df40 1 --2- 172.21.15.167:0/4024598496 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 0x55989c48cb00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc922f6df40 1 -- 172.21.15.167:0/4024598496 >> 172.21.15.167:0/4024598496 conn(0x55989c1a7400 msgr2=0x55989c2a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:18.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc922f6df40 1 -- 172.21.15.167:0/4024598496 shutdown_connections 2024-09-13T23:45:18.006 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc922f6df40 1 -- 172.21.15.167:0/4024598496 wait complete. 2024-09-13T23:45:18.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc922f6df40 1 Processor -- start 2024-09-13T23:45:18.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc922f6df40 1 -- start start 2024-09-13T23:45:18.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc922f6df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 0x55989c48cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:18.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:17.998+0000 7fc922f6df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55989c1cc8c0 con 0x55989c1a7800 2024-09-13T23:45:18.007 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.002+0000 7fc921f27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 0x55989c48cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:18.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.002+0000 7fc921f27700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 0x55989c48cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43578/0 (socket says 172.21.15.167:43578) 2024-09-13T23:45:18.008 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.002+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 learned_addr learned my addr 172.21.15.167:0/4169653719 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:18.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.002+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55989c1b8ea0 con 0x55989c1a7800 2024-09-13T23:45:18.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.002+0000 7fc921f27700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 0x55989c48cb00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55989c4c6960 tx=0x55989c510cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:18.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.002+0000 7fc91a718700 1 -- 172.21.15.167:0/4169653719 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c24ba20 con 0x55989c1a7800 2024-09-13T23:45:18.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.002+0000 7fc91a718700 1 -- 172.21.15.167:0/4169653719 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55989c56a180 con 0x55989c1a7800 2024-09-13T23:45:18.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.002+0000 7fc91a718700 1 -- 172.21.15.167:0/4169653719 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c24b600 con 0x55989c1a7800 2024-09-13T23:45:18.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.002+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55989c1b8820 con 0x55989c1a7800 2024-09-13T23:45:18.009 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.002+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55989c1b89c0 con 0x55989c1a7800 2024-09-13T23:45:18.010 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.002+0000 7fc91a718700 1 -- 172.21.15.167:0/4169653719 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55989c2a0dc0 con 0x55989c1a7800 2024-09-13T23:45:18.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.002+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc91a718700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c558c00 0x55989c48d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:18.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc91a718700 1 -- 172.21.15.167:0/4169653719 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55989c20d340 con 0x55989c1a7800 2024-09-13T23:45:18.011 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc920f25700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c558c00 0x55989c48d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:18.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55989c559000 0x55989c48db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:18.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559400 con 0x55989c559000 2024-09-13T23:45:18.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc921f27700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55989c559000 0x55989c48db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:18.012 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc920f25700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c558c00 0x55989c48d600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55989c487920 tx=0x55989c2ab800 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:18.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc921f27700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55989c559000 0x55989c48db80 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:18.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:18.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 20 realm 2024-09-13T23:45:18.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.013 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559000 2024-09-13T23:45:18.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:18.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-09-13T23:45:18.014 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559c00 con 0x55989c559000 2024-09-13T23:45:18.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:18.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55989c559400 0x55989c48e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:18.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559400 2024-09-13T23:45:18.015 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc921726700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55989c559400 0x55989c48e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:18.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.006+0000 7fc921726700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55989c559400 0x55989c48e100 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:18.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 1 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+46 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc922f6df40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:45:18.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.016 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:5 5.1b 5:d84a1873:::zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559c00 con 0x55989c559400 2024-09-13T23:45:18.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 2 ==== osd_op_reply(5 zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 [read 0~874 out=874b] v0'0 uv4 ondisk = 0) v8 ==== 190+0+874 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc922f6df40 20 rados_obj.operate() r=0 bl.length=874 2024-09-13T23:45:18.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc922f6df40 20 searching for the correct realm 2024-09-13T23:45:18.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc922f6df40 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55989c61a000 0x55989c48e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:18.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61a400 con 0x55989c61a000 2024-09-13T23:45:18.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc920f25700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55989c61a000 0x55989c48e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:18.017 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc920f25700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55989c61a000 0x55989c48e680 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:18.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 1 ==== osd_op_reply(6 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.018 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61a400 con 0x55989c559400 2024-09-13T23:45:18.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 3 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc921726700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55989c61a400 0x55989c48ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:18.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a400 2024-09-13T23:45:18.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.010+0000 7fc921f27700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55989c61a400 0x55989c48ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:18.019 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc921f27700 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55989c61a400 0x55989c48ec00 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:18.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 1 ==== osd_op_reply(8 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559000 2024-09-13T23:45:18.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 4 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.020 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559000 2024-09-13T23:45:18.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 5 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a000 2024-09-13T23:45:18.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.021 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559000 2024-09-13T23:45:18.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 6 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559400 2024-09-13T23:45:18.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 4 ==== osd_op_reply(13 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.022 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.014+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559400 2024-09-13T23:45:18.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 5 ==== osd_op_reply(14 [pgnls start_epoch 35 out=112b] v26'1 uv1 ondisk = 1) v8 ==== 144+0+112 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559000 2024-09-13T23:45:18.023 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 7 ==== osd_op_reply(15 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a400 2024-09-13T23:45:18.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 2 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559000 2024-09-13T23:45:18.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 8 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.024 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a400 2024-09-13T23:45:18.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 3 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.025 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559400 2024-09-13T23:45:18.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 6 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a000 2024-09-13T23:45:18.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.026 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559000 2024-09-13T23:45:18.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 9 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.027 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.018+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559400 2024-09-13T23:45:18.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 7 ==== osd_op_reply(22 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a000 2024-09-13T23:45:18.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.028 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559000 2024-09-13T23:45:18.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 10 ==== osd_op_reply(24 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a400 2024-09-13T23:45:18.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 4 ==== osd_op_reply(25 [pgnls start_epoch 35 out=74b] v35'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.029 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559000 2024-09-13T23:45:18.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 11 ==== osd_op_reply(26 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a000 2024-09-13T23:45:18.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.030 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a400 2024-09-13T23:45:18.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 5 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.022+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559000 2024-09-13T23:45:18.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.026+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 12 ==== osd_op_reply(29 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.031 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.026+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a400 2024-09-13T23:45:18.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.026+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 6 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.026+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a000 2024-09-13T23:45:18.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.026+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 6 ==== osd_op_reply(31 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.032 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.026+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a400 2024-09-13T23:45:18.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.026+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 7 ==== osd_op_reply(32 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.033 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.026+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559400 2024-09-13T23:45:18.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.026+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 8 ==== osd_op_reply(33 [pgnls start_epoch 35 out=107b] v35'4 uv4 ondisk = 1) v8 ==== 144+0+107 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.026+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559400 2024-09-13T23:45:18.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.026+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 9 ==== osd_op_reply(34 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.034 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.026+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a000 2024-09-13T23:45:18.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 7 ==== osd_op_reply(35 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.035 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559400 2024-09-13T23:45:18.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 10 ==== osd_op_reply(36 [pgnls start_epoch 35 out=115b] v35'3 uv3 ondisk = 1) v8 ==== 144+0+115 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.036 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559400 2024-09-13T23:45:18.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 11 ==== osd_op_reply(37 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 20 RGWRados::pool_iterate: got zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf 2024-09-13T23:45:18.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 20 RGWRados::pool_iterate: got default.zone. 2024-09-13T23:45:18.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 20 RGWRados::pool_iterate: got zone_info.9144208c-3c85-4fb6-9007-2693d5a930f0 2024-09-13T23:45:18.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 20 RGWRados::pool_iterate: got zone_names.default 2024-09-13T23:45:18.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-09-13T23:45:18.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.037 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61a800 con 0x55989c559000 2024-09-13T23:45:18.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:18.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.038 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61ac00 con 0x55989c559400 2024-09-13T23:45:18.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 12 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) v8 ==== 168+0+46 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 20 rados_obj.operate() r=0 bl.length=46 2024-09-13T23:45:18.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:40 5.2 5:47e4930b:::zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61b000 con 0x55989c559400 2024-09-13T23:45:18.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 13 ==== osd_op_reply(40 zonegroup_info.363ea8c8-5049-424d-855a-cd6979bf7acf [read 0~420 out=420b] v0'0 uv1 ondisk = 0) v8 ==== 195+0+420 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 20 rados_obj.operate() r=0 bl.length=420 2024-09-13T23:45:18.039 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 20 zone default found 2024-09-13T23:45:18.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 4 Realm: () 2024-09-13T23:45:18.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 4 ZoneGroup: default (363ea8c8-5049-424d-855a-cd6979bf7acf) 2024-09-13T23:45:18.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.030+0000 7fc922f6df40 4 Zone: default (9144208c-3c85-4fb6-9007-2693d5a930f0) 2024-09-13T23:45:18.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 10 cannot find current period zonegroup using local zonegroup configuration 2024-09-13T23:45:18.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 20 zonegroup default 2024-09-13T23:45:18.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.040 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61b400 con 0x55989c61a400 2024-09-13T23:45:18.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 165+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:18.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61b800 con 0x55989c61a000 2024-09-13T23:45:18.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 8 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:18.041 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61bc00 con 0x55989c61a000 2024-09-13T23:45:18.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 9 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 211+0+0 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:18.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 20 started sync module instance, tier type = 2024-09-13T23:45:18.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 20 started zone id=9144208c-3c85-4fb6-9007-2693d5a930f0 (name=default) with tier type = 2024-09-13T23:45:18.042 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.034+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61b800 con 0x55989c559400 2024-09-13T23:45:18.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.038+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 14 ==== osd_op_reply(44 notify.0 [create] v35'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.038+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94113945369344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61bc00 con 0x55989c559400 2024-09-13T23:45:18.044 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.038+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61ac00 con 0x55989c559000 2024-09-13T23:45:18.046 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.038+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 14 ==== osd_op_reply(46 notify.1 [create] v35'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.038+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94113945370496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61b000 con 0x55989c559000 2024-09-13T23:45:18.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.038+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61b800 con 0x55989c559400 2024-09-13T23:45:18.047 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.038+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 15 ==== osd_op_reply(45 notify.0 [watch watch cookie 94113945369344] v35'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.042+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 15 ==== osd_op_reply(47 notify.1 [watch watch cookie 94113945370496] v35'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.049 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.042+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 16 ==== osd_op_reply(48 notify.2 [create] v35'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.042+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94113945371648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559c00 con 0x55989c559400 2024-09-13T23:45:18.050 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.042+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c61a400 2024-09-13T23:45:18.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.046+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 9 ==== osd_op_reply(50 notify.3 [create] v35'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.052 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.046+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94113945372800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61a800 con 0x55989c61a400 2024-09-13T23:45:18.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.046+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61b400 con 0x55989c559400 2024-09-13T23:45:18.053 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.046+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 17 ==== osd_op_reply(49 notify.2 [watch watch cookie 94113945371648] v35'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.054 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.046+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94113945372800] v35'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.050+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 18 ==== osd_op_reply(52 notify.4 [create] v35'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.050+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94113945373952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61ac00 con 0x55989c559400 2024-09-13T23:45:18.055 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.050+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61b000 con 0x55989c61a000 2024-09-13T23:45:18.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.050+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 10 ==== osd_op_reply(54 notify.5 [create] v35'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.050+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94113945375104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c626000 con 0x55989c61a000 2024-09-13T23:45:18.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.050+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c626400 con 0x55989c61a000 2024-09-13T23:45:18.058 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.050+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 19 ==== osd_op_reply(53 notify.4 [watch watch cookie 94113945373952] v35'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 94113945375104] v35'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 12 ==== osd_op_reply(56 notify.6 [create] v35'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94113945376256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c626800 con 0x55989c61a000 2024-09-13T23:45:18.060 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c626c00 con 0x55989c61a400 2024-09-13T23:45:18.062 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 11 ==== osd_op_reply(58 notify.7 [create] v35'240 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94113945377408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c627000 con 0x55989c61a400 2024-09-13T23:45:18.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc922f6df40 20 add_watcher() i=0 2024-09-13T23:45:18.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc922f6df40 20 add_watcher() i=1 2024-09-13T23:45:18.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc922f6df40 20 add_watcher() i=2 2024-09-13T23:45:18.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc922f6df40 20 add_watcher() i=3 2024-09-13T23:45:18.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc922f6df40 20 add_watcher() i=4 2024-09-13T23:45:18.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc922f6df40 20 add_watcher() i=5 2024-09-13T23:45:18.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94113945376256] v35'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.063 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.054+0000 7fc922f6df40 20 add_watcher() i=6 2024-09-13T23:45:18.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.058+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 12 ==== osd_op_reply(59 notify.7 [watch watch cookie 94113945377408] v35'241 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.058+0000 7fc922f6df40 20 add_watcher() i=7 2024-09-13T23:45:18.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.058+0000 7fc922f6df40 2 all 8 watchers are set, enabling cache 2024-09-13T23:45:18.064 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.058+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c627400 con 0x55989c559000 2024-09-13T23:45:18.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.058+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 16 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) v8 ==== 256+0+76 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.065 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.058+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94113947107328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c627800 con 0x55989c559000 2024-09-13T23:45:18.067 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 17 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94113947107328] v35'239 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc9086f4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:45:18.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 20 check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-09-13T23:45:18.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 20 check_secure_mon_conn(): mode 1 is insecure 2024-09-13T23:45:18.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 1 Processor -- start 2024-09-13T23:45:18.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 1 -- start start 2024-09-13T23:45:18.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 0x55989c48f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:18.068 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55989c2343c0 con 0x55989c626c00 2024-09-13T23:45:18.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc920f25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 0x55989c48f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:18.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc920f25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 0x55989c48f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43582/0 (socket says 172.21.15.167:43582) 2024-09-13T23:45:18.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc920f25700 1 -- 172.21.15.167:0/1496857071 learned_addr learned my addr 172.21.15.167:0/1496857071 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:18.069 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc920f25700 1 -- 172.21.15.167:0/1496857071 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55989c1b8b60 con 0x55989c626c00 2024-09-13T23:45:18.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc920f25700 1 --2- 172.21.15.167:0/1496857071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 0x55989c48f180 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x55989c6340f0 tx=0x55989c2ab860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4691f99e98ef924d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:18.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc907ef3700 1 -- 172.21.15.167:0/1496857071 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c24b760 con 0x55989c626c00 2024-09-13T23:45:18.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc907ef3700 1 -- 172.21.15.167:0/1496857071 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55989c56a600 con 0x55989c626c00 2024-09-13T23:45:18.070 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc907ef3700 1 -- 172.21.15.167:0/1496857071 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c63e000 con 0x55989c626c00 2024-09-13T23:45:18.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 1 -- 172.21.15.167:0/1496857071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 msgr2=0x55989c48f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:18.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 1 --2- 172.21.15.167:0/1496857071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 0x55989c48f180 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x55989c6340f0 tx=0x55989c2ab860 comp rx=0 tx=0).stop 2024-09-13T23:45:18.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 1 -- 172.21.15.167:0/1496857071 shutdown_connections 2024-09-13T23:45:18.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 1 --2- 172.21.15.167:0/1496857071 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 0x55989c48f180 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.071 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 1 -- 172.21.15.167:0/1496857071 >> 172.21.15.167:0/1496857071 conn(0x55989c627c00 msgr2=0x55989c2a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:18.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 1 -- 172.21.15.167:0/1496857071 shutdown_connections 2024-09-13T23:45:18.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 1 -- 172.21.15.167:0/1496857071 wait complete. 2024-09-13T23:45:18.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.062+0000 7fc922f6df40 1 Processor -- start 2024-09-13T23:45:18.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc922f6df40 1 -- start start 2024-09-13T23:45:18.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc922f6df40 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 0x55989c48f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:18.072 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc922f6df40 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x55989c234500 con 0x55989c626c00 2024-09-13T23:45:18.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc920f25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 0x55989c48f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:18.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc920f25700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 0x55989c48f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.167:43594/0 (socket says 172.21.15.167:43594) 2024-09-13T23:45:18.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc920f25700 1 -- 172.21.15.167:0/169197871 learned_addr learned my addr 172.21.15.167:0/169197871 (peer_addr_for_me v2:172.21.15.167:0/0) 2024-09-13T23:45:18.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc920f25700 1 -- 172.21.15.167:0/169197871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55989c1b8d00 con 0x55989c626c00 2024-09-13T23:45:18.073 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc920f25700 1 --2- 172.21.15.167:0/169197871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 0x55989c48f180 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x55989c634390 tx=0x55989c604420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:18.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc9066f0700 1 -- 172.21.15.167:0/169197871 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c63e160 con 0x55989c626c00 2024-09-13T23:45:18.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc9066f0700 1 -- 172.21.15.167:0/169197871 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55989c56a300 con 0x55989c626c00 2024-09-13T23:45:18.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc9066f0700 1 -- 172.21.15.167:0/169197871 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x55989c63e2c0 con 0x55989c626c00 2024-09-13T23:45:18.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc922f6df40 1 -- 172.21.15.167:0/169197871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55989c1b8340 con 0x55989c626c00 2024-09-13T23:45:18.074 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc922f6df40 1 -- 172.21.15.167:0/169197871 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55989c1b9d40 con 0x55989c626c00 2024-09-13T23:45:18.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc9066f0700 1 -- 172.21.15.167:0/169197871 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x55989c2a1b80 con 0x55989c626c00 2024-09-13T23:45:18.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc9066f0700 1 --2- 172.21.15.167:0/169197871 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c6b0000 0x55989c6b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:45:18.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc9066f0700 1 -- 172.21.15.167:0/169197871 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x55989c20d500 con 0x55989c626c00 2024-09-13T23:45:18.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc921f27700 1 --2- 172.21.15.167:0/169197871 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c6b0000 0x55989c6b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:45:18.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.066+0000 7fc922f6df40 5 note: GC not initialized 2024-09-13T23:45:18.075 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc921f27700 1 --2- 172.21.15.167:0/169197871 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c6b0000 0x55989c6b6000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55989c5b7ce0 tx=0x55989c5117a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:45:18.077 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 20 init_complete bucket index max shards: 11 2024-09-13T23:45:18.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc903eeb700 20 reqs_thread_entry: start 2024-09-13T23:45:18.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 10 rgw notify: Started notification manager with: 1 workers 2024-09-13T23:45:18.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc901ee7700 20 rgw notify: INFO: next queues processing will happen at: Fri Sep 13 23:45:48 2024 2024-09-13T23:45:18.078 INFO:teuthology.orchestra.run.smithi167.stderr: 2024-09-13T23:45:18.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-09-13T23:45:18.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c626000 con 0x55989c559000 2024-09-13T23:45:18.078 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc9016c5700 20 reqs_thread_entry: start 2024-09-13T23:45:18.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 18 ==== osd_op_reply(62 ada [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 231+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:18.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x0 2024-09-13T23:45:18.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-09-13T23:45:18.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : miss 2024-09-13T23:45:18.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 20 rados->read ofs=0 len=0 2024-09-13T23:45:18.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:63 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c626400 con 0x55989c559000 2024-09-13T23:45:18.080 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 19 ==== osd_op_reply(63 MH1WC2XQ1S8UISFDZC8W [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 206+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 20 rados_obj.operate() r=-2 bl.length=0 2024-09-13T23:45:18.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x0 2024-09-13T23:45:18.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 10 adding default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-09-13T23:45:18.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-09-13T23:45:18.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-09-13T23:45:18.081 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.070+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:64 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.set in=58b,writefull 0~439 in=439b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61a800 con 0x55989c559000 2024-09-13T23:45:18.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.074+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 20 ==== osd_op_reply(64 ada [delete,create,call,writefull 0~439] v35'3 uv3 ondisk = 0) v8 ==== 273+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.074+0000 7fc922f6df40 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-13T23:45:18.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.074+0000 7fc922f6df40 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-13T23:45:18.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.074+0000 7fc922f6df40 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-09-13T23:45:18.082 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.074+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94113947109632 in=627b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61b000 con 0x55989c559400 2024-09-13T23:45:18.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.074+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 20 ==== watch-notify(notify (1) cookie 94113945373952 notify 150323855406 ret 0) v3 ==== 657+0+0 (crc 0 0 0) 0x55989c50e820 con 0x55989c559400 2024-09-13T23:45:18.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.074+0000 7fc91b71a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855406 cookie 94113945373952 notifier 4910 bl.length()=615 2024-09-13T23:45:18.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.074+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 21 ==== osd_op_reply(65 notify.4 [notify cookie 94113947109632 out=8b] v0'0 uv240 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.074+0000 7fc91b71a700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-09-13T23:45:18.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.074+0000 7fc91b71a700 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-09-13T23:45:18.083 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.074+0000 7fc91b71a700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61bc00 con 0x55989c559400 2024-09-13T23:45:18.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 22 ==== watch-notify(notify_complete (2) cookie 94113947109632 notify 150323855406 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55989c50e9c0 con 0x55989c559400 2024-09-13T23:45:18.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 23 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.084 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:67 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559800 con 0x55989c559000 2024-09-13T23:45:18.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 21 ==== osd_op_reply(67 MH1WC2XQ1S8UISFDZC8W [delete,create,writefull 0~7] v35'21 uv21 ondisk = 0) v8 ==== 248+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc922f6df40 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-09-13T23:45:18.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc922f6df40 10 moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-09-13T23:45:18.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc922f6df40 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:MH1WC2XQ1S8UISFDZC8W, ofs0, ns] 2024-09-13T23:45:18.086 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94113947109632 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c6b0400 con 0x55989c559400 2024-09-13T23:45:18.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 24 ==== watch-notify(notify (1) cookie 94113945371648 notify 150323855407 ret 0) v3 ==== 219+0+0 (crc 0 0 0) 0x55989c50e9c0 con 0x55989c559400 2024-09-13T23:45:18.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc91b71a700 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 150323855407 cookie 94113945371648 notifier 4910 bl.length()=177 2024-09-13T23:45:18.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc91b71a700 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-09-13T23:45:18.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 25 ==== osd_op_reply(68 notify.2 [notify cookie 94113947109632 out=8b] v0'0 uv240 ondisk = 0) v8 ==== 152+0+8 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc91b71a700 10 rgw watcher librados: moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-09-13T23:45:18.087 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.078+0000 7fc91b71a700 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c559c00 con 0x55989c559400 2024-09-13T23:45:18.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.082+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 26 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.088 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.082+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 27 ==== watch-notify(notify_complete (2) cookie 94113947109632 notify 150323855407 ret 0) v3 ==== 42+0+48 (crc 0 0 0) 0x55989c50eb60 con 0x55989c559400 2024-09-13T23:45:18.088 INFO:teuthology.orchestra.run.smithi167.stdout:{ 2024-09-13T23:45:18.088 INFO:teuthology.orchestra.run.smithi167.stdout: "user_id": "ada", 2024-09-13T23:45:18.088 INFO:teuthology.orchestra.run.smithi167.stdout: "display_name": "Ms. Admin User", 2024-09-13T23:45:18.088 INFO:teuthology.orchestra.run.smithi167.stdout: "email": "", 2024-09-13T23:45:18.088 INFO:teuthology.orchestra.run.smithi167.stdout: "suspended": 0, 2024-09-13T23:45:18.088 INFO:teuthology.orchestra.run.smithi167.stdout: "max_buckets": 0, 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: "subusers": [], 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: "keys": [ 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: "user": "ada", 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: "access_key": "MH1WC2XQ1S8UISFDZC8W", 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: "secret_key": "dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG" 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: "swift_keys": [], 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: "caps": [ 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "buckets", 2024-09-13T23:45:18.089 INFO:teuthology.orchestra.run.smithi167.stdout: "perm": "*" 2024-09-13T23:45:18.090 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:18.090 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:18.090 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "info", 2024-09-13T23:45:18.090 INFO:teuthology.orchestra.run.smithi167.stdout: "perm": "read" 2024-09-13T23:45:18.090 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:18.090 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:18.091 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "ratelimit", 2024-09-13T23:45:18.091 INFO:teuthology.orchestra.run.smithi167.stdout: "perm": "*" 2024-09-13T23:45:18.091 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:18.091 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:18.091 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "usage", 2024-09-13T23:45:18.091 INFO:teuthology.orchestra.run.smithi167.stdout: "perm": "*" 2024-09-13T23:45:18.092 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:18.092 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:18.092 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "users", 2024-09-13T23:45:18.092 INFO:teuthology.orchestra.run.smithi167.stdout: "perm": "*" 2024-09-13T23:45:18.092 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:18.092 INFO:teuthology.orchestra.run.smithi167.stdout: { 2024-09-13T23:45:18.093 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "zone", 2024-09-13T23:45:18.093 INFO:teuthology.orchestra.run.smithi167.stdout: "perm": "*" 2024-09-13T23:45:18.093 INFO:teuthology.orchestra.run.smithi167.stdout: } 2024-09-13T23:45:18.093 INFO:teuthology.orchestra.run.smithi167.stdout: ], 2024-09-13T23:45:18.093 INFO:teuthology.orchestra.run.smithi167.stdout: "op_mask": "read, write, delete", 2024-09-13T23:45:18.093 INFO:teuthology.orchestra.run.smithi167.stdout: "default_placement": "", 2024-09-13T23:45:18.094 INFO:teuthology.orchestra.run.smithi167.stdout: "default_storage_class": "", 2024-09-13T23:45:18.094 INFO:teuthology.orchestra.run.smithi167.stdout: "placement_tags": [], 2024-09-13T23:45:18.094 INFO:teuthology.orchestra.run.smithi167.stdout: "bucket_quota": { 2024-09-13T23:45:18.094 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:45:18.094 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:45:18.094 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:45:18.094 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:45:18.095 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:45:18.095 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:18.095 INFO:teuthology.orchestra.run.smithi167.stdout: "user_quota": { 2024-09-13T23:45:18.095 INFO:teuthology.orchestra.run.smithi167.stdout: "enabled": false, 2024-09-13T23:45:18.095 INFO:teuthology.orchestra.run.smithi167.stdout: "check_on_raw": false, 2024-09-13T23:45:18.095 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size": -1, 2024-09-13T23:45:18.095 INFO:teuthology.orchestra.run.smithi167.stdout: "max_size_kb": 0, 2024-09-13T23:45:18.095 INFO:teuthology.orchestra.run.smithi167.stdout: "max_objects": -1 2024-09-13T23:45:18.096 INFO:teuthology.orchestra.run.smithi167.stdout: }, 2024-09-13T23:45:18.096 INFO:teuthology.orchestra.run.smithi167.stdout: "temp_url_keys": [], 2024-09-13T23:45:18.096 INFO:teuthology.orchestra.run.smithi167.stdout: "type": "rgw", 2024-09-13T23:45:18.096 INFO:teuthology.orchestra.run.smithi167.stdout: "mfa_ids": [] 2024-09-13T23:45:18.096 INFO:teuthology.orchestra.run.smithi167.stdout:} 2024-09-13T23:45:18.096 INFO:teuthology.orchestra.run.smithi167.stdout: 2024-09-13T23:45:18.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.082+0000 7fc9086f4700 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-09-13T23:45:18.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.082+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94113947107328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c6b0800 con 0x55989c559000 2024-09-13T23:45:18.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.086+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94113947107328] v35'240 uv1 ondisk = 0) v8 ==== 172+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.086+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94113945369344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c6b0c00 con 0x55989c559400 2024-09-13T23:45:18.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.086+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 28 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94113945369344] v35'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.099 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.086+0000 7fc922f6df40 20 remove_watcher() i=0 2024-09-13T23:45:18.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.090+0000 7fc922f6df40 2 removed watcher, disabling cache 2024-09-13T23:45:18.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.090+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94113945370496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c6b1000 con 0x55989c559000 2024-09-13T23:45:18.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.090+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.1 v2:172.21.15.167:6800/3875661733 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94113945370496] v35'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c559000 2024-09-13T23:45:18.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.090+0000 7fc922f6df40 20 remove_watcher() i=1 2024-09-13T23:45:18.100 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.090+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94113945371648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c6b1400 con 0x55989c559400 2024-09-13T23:45:18.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.094+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 29 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94113945371648] v35'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.094+0000 7fc922f6df40 20 remove_watcher() i=2 2024-09-13T23:45:18.101 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.094+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94113945372800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c6b1800 con 0x55989c61a400 2024-09-13T23:45:18.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.094+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 13 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94113945372800] v35'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.094+0000 7fc922f6df40 20 remove_watcher() i=3 2024-09-13T23:45:18.102 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.094+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94113945373952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c6b1c00 con 0x55989c559400 2024-09-13T23:45:18.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.098+0000 7fc921726700 1 -- 172.21.15.167:0/4169653719 <== osd.0 v2:172.21.15.12:6800/2649399468 30 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94113945373952] v35'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611200 con 0x55989c559400 2024-09-13T23:45:18.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.098+0000 7fc922f6df40 20 remove_watcher() i=4 2024-09-13T23:45:18.105 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.098+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94113945375104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61b800 con 0x55989c61a000 2024-09-13T23:45:18.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.102+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 14 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94113945375104] v35'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.102+0000 7fc922f6df40 20 remove_watcher() i=5 2024-09-13T23:45:18.107 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.102+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94113945376256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61b400 con 0x55989c61a000 2024-09-13T23:45:18.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.102+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 <== osd.3 v2:172.21.15.167:6816/3688363524 15 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94113945376256] v35'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c611440 con 0x55989c61a000 2024-09-13T23:45:18.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.102+0000 7fc922f6df40 20 remove_watcher() i=6 2024-09-13T23:45:18.110 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.102+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 --> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94113945377408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35) v8 -- 0x55989c61ac00 con 0x55989c61a400 2024-09-13T23:45:18.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc921f27700 1 -- 172.21.15.167:0/4169653719 <== osd.2 v2:172.21.15.167:6808/1180369139 14 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94113945377408] v35'242 uv240 ondisk = 0) v8 ==== 152+0+0 (crc 0 0 0) 0x55989c610b40 con 0x55989c61a400 2024-09-13T23:45:18.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 20 remove_watcher() i=7 2024-09-13T23:45:18.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55989c559400 msgr2=0x55989c48e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:18.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.12:6800/2649399468,v1:172.21.15.12:6801/2649399468] conn(0x55989c559400 0x55989c48e100 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55989c559000 msgr2=0x55989c48db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:18.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6800/3875661733,v1:172.21.15.167:6801/3875661733] conn(0x55989c559000 0x55989c48db80 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55989c61a400 msgr2=0x55989c48ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:18.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6808/1180369139,v1:172.21.15.167:6809/1180369139] conn(0x55989c61a400 0x55989c48ec00 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55989c61a000 msgr2=0x55989c48e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:18.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6816/3688363524,v1:172.21.15.167:6817/3688363524] conn(0x55989c61a000 0x55989c48e680 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c558c00 msgr2=0x55989c48d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:18.113 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c558c00 0x55989c48d600 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55989c487920 tx=0x55989c2ab800 comp rx=0 tx=0).stop 2024-09-13T23:45:18.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 msgr2=0x55989c48cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:18.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/4169653719 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7800 0x55989c48cb00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55989c4c6960 tx=0x55989c510cc0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 reap_dead start 2024-09-13T23:45:18.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc920f25700 1 -- 172.21.15.167:0/4169653719 reap_dead start 2024-09-13T23:45:18.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 shutdown_connections 2024-09-13T23:45:18.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 >> 172.21.15.167:0/4169653719 conn(0x55989c1a7400 msgr2=0x55989c2a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:18.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 shutdown_connections 2024-09-13T23:45:18.114 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/4169653719 wait complete. 2024-09-13T23:45:18.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/169197871 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c6b0000 msgr2=0x55989c6b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:18.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/169197871 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c6b0000 0x55989c6b6000 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55989c5b7ce0 tx=0x55989c5117a0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/169197871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 msgr2=0x55989c48f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:18.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/169197871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 0x55989c48f180 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x55989c634390 tx=0x55989c604420 comp rx=0 tx=0).stop 2024-09-13T23:45:18.116 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/169197871 shutdown_connections 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/169197871 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c6b0000 0x55989c6b6000 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/169197871 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c626c00 0x55989c48f180 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/169197871 >> 172.21.15.167:0/169197871 conn(0x55989c627c00 msgr2=0x55989c2a8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/169197871 shutdown_connections 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/169197871 wait complete. 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/3453674734 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c558000 msgr2=0x55989c48d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/3453674734 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c558000 0x55989c48d080 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55989c4c7110 tx=0x55989c510240 comp rx=0 tx=0).stop 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/3453674734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 msgr2=0x55989c48c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/3453674734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 0x55989c48c580 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x55989d015f80 tx=0x55989c2aba40 comp rx=0 tx=0).stop 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/3453674734 shutdown_connections 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/3453674734 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x55989c558000 0x55989c48d080 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 --2- 172.21.15.167:0/3453674734 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x55989c1a7000 0x55989c48c580 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:45:18.117 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/3453674734 >> 172.21.15.167:0/3453674734 conn(0x55989c1a6c00 msgr2=0x55989c2a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:45:18.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/3453674734 shutdown_connections 2024-09-13T23:45:18.118 INFO:teuthology.orchestra.run.smithi167.stderr:2024-09-13T23:45:18.106+0000 7fc922f6df40 1 -- 172.21.15.167:0/3453674734 wait complete. 2024-09-13T23:45:18.120 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.120 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.120 ERROR:root:0 2024-09-13T23:45:18.121 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.158 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx00000d69917c85f6fe611-0066e4ce8e-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:45:18.158 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'foo', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '4'} 2024-09-13T23:45:18.188 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.188 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 0} 2024-09-13T23:45:18.226 INFO:tasks.radosgw_admin_rest: json result: {'keys': [], 'truncated': True, 'count': 0, 'marker': '8:1e6ca5ed:users.uid::foo:head'} 2024-09-13T23:45:18.227 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1} 2024-09-13T23:45:18.264 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['foo'], 'truncated': True, 'count': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-09-13T23:45:18.264 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-09-13T23:45:18.286 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['ada'], 'truncated': False, 'count': 1} 2024-09-13T23:45:18.286 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.298 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.299 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'access-key': '9te6NH5mcdcq0Tc5i8i1', 'uid': 'uid_not_exist'} 2024-09-13T23:45:18.312 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.312 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-09-13T23:45:18.330 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.330 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.342 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.343 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-09-13T23:45:18.360 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.361 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.373 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.373 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'create'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'} 2024-09-13T23:45:18.396 INFO:tasks.radosgw_admin_rest: json result: [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}] 2024-09-13T23:45:18.397 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.409 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.409 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ'} 2024-09-13T23:45:18.431 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:45:18.431 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.461 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.461 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo1', 'secret-key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'key-type': 'swift'} 2024-09-13T23:45:18.485 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}] 2024-09-13T23:45:18.486 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.498 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.498 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo2', 'secret-key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'key-type': 'swift'} 2024-09-13T23:45:18.521 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}] 2024-09-13T23:45:18.521 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.534 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.534 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'subuser': 'foo:foo1', 'key-type': 'swift'} 2024-09-13T23:45:18.556 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:45:18.557 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.569 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.569 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo1'} 2024-09-13T23:45:18.590 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:45:18.590 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.602 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy'}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.602 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo2', 'key-type': 'swift', '{purge-keys': True} 2024-09-13T23:45:18.623 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:45:18.624 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.636 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.636 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.649 INFO:tasks.radosgw_admin_rest: json result: [] 2024-09-13T23:45:18.650 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-09-13T23:45:18.663 INFO:tasks.radosgw_admin_rest: json result: [] 2024-09-13T23:45:18.692 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-09-13T23:45:18.705 INFO:tasks.radosgw_admin_rest: json result: ['myfoo'] 2024-09-13T23:45:18.706 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-09-13T23:45:18.719 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '363ea8c8-5049-424d-855a-cd6979bf7acf', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.9', 'marker': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.9', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-13T23:45:18.682744Z', 'creation_time': '2024-09-13T23:45:18.676525Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-09-13T23:45:18.720 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-09-13T23:45:18.735 INFO:tasks.radosgw_admin_rest: json result: [{'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '363ea8c8-5049-424d-855a-cd6979bf7acf', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.9', 'marker': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.9', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-13T23:45:18.682744Z', 'creation_time': '2024-09-13T23:45:18.676525Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2024-09-13T23:45:18.745 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-09-13T23:45:18.759 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '363ea8c8-5049-424d-855a-cd6979bf7acf', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.9', 'marker': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.9', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-13T23:45:18.682744Z', 'creation_time': '2024-09-13T23:45:18.676525Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-09-13T23:45:18.760 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'doesnotexist'} 2024-09-13T23:45:18.772 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchBucket', 'Message': '', 'RequestId': 'tx00000e851ca2e821dfd60-0066e4ce8e-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:45:18.781 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'unlink'] {'uid': 'foo', 'bucket': 'myfoo'} 2024-09-13T23:45:18.800 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:45:18.800 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'fud', 'display-name': 'Fud', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'max-buckets': '1'} 2024-09-13T23:45:18.824 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:45:18.842 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'fud', 'bucket': 'myfoo', 'bucket-id': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.9'} 2024-09-13T23:45:18.867 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:45:18.913 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'foo', 'bucket': 'myfoo', 'bucket-id': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.9'} 2024-09-13T23:45:18.941 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:45:18.942 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'fud'} 2024-09-13T23:45:18.968 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:45:18.991 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['object', 'rm'] {'bucket': 'myfoo', 'object': 'four'} 2024-09-13T23:45:19.010 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:45:19.010 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-09-13T23:45:19.025 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'num_shards': 11, 'tenant': '', 'zonegroup': '363ea8c8-5049-424d-855a-cd6979bf7acf', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.9', 'marker': '9144208c-3c85-4fb6-9007-2693d5a930f0.4224.9', 'index_type': 'Normal', 'versioned': False, 'versioning_enabled': False, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-09-13T23:45:18.928877Z', 'creation_time': '2024-09-13T23:45:18.676525Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-09-13T23:46:19.128 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'categories': 'delete_obj'} 2024-09-13T23:46:19.168 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'anonymous', 'categories': []}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'bar', 'categories': []}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'bud', 'categories': []}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}, {'bucket': 'useless-bucket', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'fud', 'categories': []}, {'bucket': 'myfoo7', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'fud', 'categories': []}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'bar', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'bud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}}, {'user': 'fud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}]} 2024-09-13T23:46:19.169 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] None 2024-09-13T23:46:19.206 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'fud', 'categories': [{'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 0, 'ops': 1, 'successful_ops': 0}]}, {'bucket': 'myfoo7', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 231, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}, {'user': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 8, 'ops': 2, 'successful_ops': 2}}, {'user': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 16, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10}}, {'user': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 8, 'ops': 2, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 242, 'bytes_received': 8, 'ops': 5, 'successful_ops': 4}}]} 2024-09-13T23:46:19.207 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-09-13T23:46:19.223 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10}}]} 2024-09-13T23:46:19.223 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'create_bucket'} 2024-09-13T23:46:19.238 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'useless-bucket', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}}]} 2024-09-13T23:46:19.238 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'put_obj'} 2024-09-13T23:46:19.251 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}}]} 2024-09-13T23:46:19.252 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_obj'} 2024-09-13T23:46:19.265 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}, {'bucket': 'useless-bucket', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}}]} 2024-09-13T23:46:19.265 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_bucket'} 2024-09-13T23:46:19.278 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': []}, {'bucket': 'useless-bucket', 'time': '2024-09-13T23:00:00.000000Z', 'epoch': 1726268400, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}]} 2024-09-13T23:46:19.279 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'trim'] {'uid': 'foo'} 2024-09-13T23:46:19.297 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:46:19.298 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-09-13T23:46:19.311 INFO:tasks.radosgw_admin_rest: json result: {'entries': [], 'summary': []} 2024-09-13T23:46:19.312 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-09-13T23:46:19.336 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:46:19.350 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-09-13T23:46:19.374 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:46:19.699 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-09-13T23:46:19.712 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx0000064b4168c4399d801-0066e4cecb-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:46:19.888 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-09-13T23:46:19.901 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [], 'grant_map': [{'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'email': '', 'permission': {'flags': 15}, 'name': 'Foo', 'group': 0, 'url_spec': ''}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-09-13T23:46:19.911 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-09-13T23:46:19.924 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [{'group': 1, 'acl': 1}], 'grant_map': [{'id': '', 'grant': {'type': {'type': 2}, 'id': '', 'email': '', 'permission': {'flags': 1}, 'name': '', 'group': 1, 'url_spec': ''}}, {'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'email': '', 'permission': {'flags': 15}, 'name': 'Foo', 'group': 0, 'url_spec': ''}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-09-13T23:46:20.037 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'rm'] {'bucket': 'myfoo', 'purge-objects': True} 2024-09-13T23:46:20.128 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:46:20.128 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'add'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-09-13T23:46:20.145 INFO:tasks.radosgw_admin_rest: json result: [{'type': 'usage', 'perm': 'read'}] 2024-09-13T23:46:20.145 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'rm'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-09-13T23:46:20.162 INFO:tasks.radosgw_admin_rest: json result: [] 2024-09-13T23:46:20.181 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-09-13T23:46:20.193 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx0000041f17f9ca41d42f9-0066e4cecc-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:46:20.207 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo', 'purge-data': True} 2024-09-13T23:46:20.286 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:46:20.286 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-09-13T23:46:20.299 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx00000a7e7e953b571e60d-0066e4cecc-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:46:20.299 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['info', ''] None 2024-09-13T23:46:20.311 INFO:tasks.radosgw_admin_rest: json result: {'info': {'storage_backends': [{'name': 'rados', 'cluster_id': 'dc64727a-88fa-459b-b2f7-e6853bb706a5'}]}} 2024-09-13T23:46:20.311 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'ratelimit_user', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '1000'} 2024-09-13T23:46:20.340 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'ratelimit_user', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'ratelimit_user', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-09-13T23:46:20.340 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-09-13T23:46:20.352 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-09-13T23:46:20.352 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring'} 2024-09-13T23:46:20.366 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000cc48500a8ddfe490-0066e4cecc-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:46:20.366 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user'} 2024-09-13T23:46:20.378 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000a841457b896f01f9-0066e4cecc-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:46:20.400 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-09-13T23:46:20.412 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-09-13T23:46:20.413 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring'} 2024-09-13T23:46:20.426 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000ad3439e74abcb09f-0066e4cecc-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:46:20.426 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket'} 2024-09-13T23:46:20.438 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx000001e5871a587b92cc3-0066e4cecc-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:46:20.438 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'global': 'true'} 2024-09-13T23:46:20.451 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'anonymous_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-09-13T23:46:20.451 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring', 'enabled': 'true'} 2024-09-13T23:46:20.464 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000f34d30f2d4604af1-0066e4cecc-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:46:20.464 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user'} 2024-09-13T23:46:20.476 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx000006a2b7957efaad095-0066e4cecc-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:46:20.476 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring', 'enabled': 'true'} 2024-09-13T23:46:20.487 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx000003d8f4ac89eb13b8b-0066e4cecc-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:46:20.488 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'enabled': 'true'} 2024-09-13T23:46:20.499 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000d991ba4df6793434-0066e4cecc-107e-default', 'HostId': '107e-default-default'} 2024-09-13T23:46:20.500 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user', 'enabled': 'true', 'max-read-bytes': '2'} 2024-09-13T23:46:20.526 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:46:20.526 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-09-13T23:46:20.538 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 2, 'max_write_bytes': 0, 'enabled': True}} 2024-09-13T23:46:20.539 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket', 'enabled': 'true', 'max-write-bytes': '2'} 2024-09-13T23:46:20.557 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:46:20.557 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-09-13T23:46:20.569 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 2, 'enabled': True}} 2024-09-13T23:46:20.569 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'global': 'true', 'enabled': 'true'} 2024-09-13T23:46:20.587 INFO:tasks.radosgw_admin_rest: empty result 2024-09-13T23:46:20.592 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-09-13T23:46:20.601 DEBUG:tasks.rgw.client.0:waiting for process to exit 2024-09-13T23:46:20.601 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T23:46:20.602 INFO:tasks.rgw.client.0.smithi167.stdout:2024-09-13T23:46:20.594+0000 7ff5dd762700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term radosgw --rgw-frontends beast port=80 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 24371) UID: 0 2024-09-13T23:46:20.603 INFO:tasks.rgw.client.0.smithi167.stdout:2024-09-13T23:46:20.594+0000 7ff5e0fc1f80 -1 shutting down 2024-09-13T23:46:25.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2024-09-13T23:46:26.703 INFO:tasks.rgw.client.0:Stopped 2024-09-13T23:46:26.703 DEBUG:teuthology.orchestra.run.smithi167:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2024-09-13T23:46:26.711 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /etc/ceph/vault-root-token 2024-09-13T23:46:26.769 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f /home/ubuntu/cephtest/url_file 2024-09-13T23:46:26.781 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T23:46:26.796 DEBUG:teuthology.orchestra.run.smithi012:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T23:46:26.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbbd424700 1 Processor -- start 2024-09-13T23:46:26.924 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbbd424700 1 -- start start 2024-09-13T23:46:26.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbbd424700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdbb812db70 0x7fdbb81325b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:46:26.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbbd424700 1 -- --> v1:172.21.15.12:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb806f900 con 0x7fdbb812d080 2024-09-13T23:46:26.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbbd424700 1 -- --> v2:172.21.15.12:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb8070fe0 con 0x7fdbb812db70 2024-09-13T23:46:26.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbb77fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdbb812db70 0x7fdbb81325b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:46:26.925 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbb77fe700 1 --2- >> v2:172.21.15.12:3300/0 conn(0x7fdbb812db70 0x7fdbb81325b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45912/0 (socket says 172.21.15.12:45912) 2024-09-13T23:46:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbb77fe700 1 -- 172.21.15.12:0/2467475172 learned_addr learned my addr 172.21.15.12:0/2467475172 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:46:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbb7fff700 1 --1- >> v1:172.21.15.12:6789/0 conn(0x7fdbb812d080 0x7fdbb812d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.12:6789/0 says I am v1:172.21.15.12:46212/0 (socket says 172.21.15.12:46212) 2024-09-13T23:46:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbb77fe700 1 -- 172.21.15.12:0/2467475172 >> v1:172.21.15.12:6789/0 conn(0x7fdbb812d080 legacy=0x7fdbb812d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:46:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbb77fe700 1 -- 172.21.15.12:0/2467475172 --> v2:172.21.15.12:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb809d0f0 con 0x7fdbb812db70 2024-09-13T23:46:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbb77fe700 1 --2- 172.21.15.12:0/2467475172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbb812db70 0x7fdbb81325b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009890 tx=0x7fdba0009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980cf9ec17a669ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:46:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbb6ffd700 1 -- 172.21.15.12:0/2467475172 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdba0004160 con 0x7fdbb812db70 2024-09-13T23:46:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbb6ffd700 1 -- 172.21.15.12:0/2467475172 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba00042c0 con 0x7fdbb812db70 2024-09-13T23:46:26.926 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.920+0000 7fdbb6ffd700 1 -- 172.21.15.12:0/2467475172 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdba0004580 con 0x7fdbb812db70 2024-09-13T23:46:26.927 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 -- 172.21.15.12:0/2467475172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbb812db70 msgr2=0x7fdbb81325b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:46:26.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 --2- 172.21.15.12:0/2467475172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbb812db70 0x7fdbb81325b0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009890 tx=0x7fdba0009ba0 comp rx=0 tx=0).stop 2024-09-13T23:46:26.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 -- 172.21.15.12:0/2467475172 shutdown_connections 2024-09-13T23:46:26.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 --2- 172.21.15.12:0/2467475172 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbb812db70 0x7fdbb81325b0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:46:26.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 -- 172.21.15.12:0/2467475172 >> 172.21.15.12:0/2467475172 conn(0x7fdbb809be20 msgr2=0x7fdbb809c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:46:26.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 -- 172.21.15.12:0/2467475172 shutdown_connections 2024-09-13T23:46:26.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 -- 172.21.15.12:0/2467475172 wait complete. 2024-09-13T23:46:26.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 Processor -- start 2024-09-13T23:46:26.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 -- start start 2024-09-13T23:46:26.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbb812d080 0x7fdbb811baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:46:26.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 -- --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb811d3b0 con 0x7fdbb812d080 2024-09-13T23:46:26.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbb7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbb812d080 0x7fdbb811baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:46:26.928 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbb7fff700 1 --2- >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbb812d080 0x7fdbb811baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.12:3300/0 says I am v2:172.21.15.12:45922/0 (socket says 172.21.15.12:45922) 2024-09-13T23:46:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbb7fff700 1 -- 172.21.15.12:0/649533180 learned_addr learned my addr 172.21.15.12:0/649533180 (peer_addr_for_me v2:172.21.15.12:0/0) 2024-09-13T23:46:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbb7fff700 1 -- 172.21.15.12:0/649533180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba0009570 con 0x7fdbb812d080 2024-09-13T23:46:26.929 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbb7fff700 1 --2- 172.21.15.12:0/649533180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbb812d080 0x7fdbb811baf0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdba800c480 tx=0x7fdba800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:46:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbb4ff9700 1 -- 172.21.15.12:0/649533180 <== mon.0 v2:172.21.15.12:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdba8005c00 con 0x7fdbb812d080 2024-09-13T23:46:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbb4ff9700 1 -- 172.21.15.12:0/649533180 <== mon.0 v2:172.21.15.12:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba8005d60 con 0x7fdbb812d080 2024-09-13T23:46:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbb4ff9700 1 -- 172.21.15.12:0/649533180 <== mon.0 v2:172.21.15.12:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdba80070d0 con 0x7fdbb812d080 2024-09-13T23:46:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 -- 172.21.15.12:0/649533180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb811d8c0 con 0x7fdbb812d080 2024-09-13T23:46:26.930 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.924+0000 7fdbbd424700 1 -- 172.21.15.12:0/649533180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb811e630 con 0x7fdbb812d080 2024-09-13T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.928+0000 7fdbb4ff9700 1 -- 172.21.15.12:0/649533180 <== mon.0 v2:172.21.15.12:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fdba8007230 con 0x7fdbb812d080 2024-09-13T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.928+0000 7fdbbd424700 1 -- 172.21.15.12:0/649533180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb811e8e0 con 0x7fdbb812d080 2024-09-13T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.928+0000 7fdbb4ff9700 1 --2- 172.21.15.12:0/649533180 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fdb980354f0 0x7fdb980379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.928+0000 7fdbb4ff9700 1 -- 172.21.15.12:0/649533180 <== mon.0 v2:172.21.15.12:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6264+0+0 (secure 0 0 0) 0x7fdba8047580 con 0x7fdbb812d080 2024-09-13T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.928+0000 7fdbb77fe700 1 --2- 172.21.15.12:0/649533180 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fdb980354f0 0x7fdb980379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T23:46:26.931 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.928+0000 7fdbb77fe700 1 --2- 172.21.15.12:0/649533180 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fdb980354f0 0x7fdb980379b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009fd0 tx=0x7fdba0008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T23:46:26.938 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:26.936+0000 7fdbb4ff9700 1 -- 172.21.15.12:0/649533180 <== mon.0 v2:172.21.15.12:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdba8019890 con 0x7fdbb812d080 2024-09-13T23:46:27.115 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.112+0000 7fdbbd424700 1 -- 172.21.15.12:0/649533180 --> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fdbb812d460 con 0x7fdbb812d080 2024-09-13T23:46:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.116+0000 7fdbb4ff9700 1 -- 172.21.15.12:0/649533180 <== mon.0 v2:172.21.15.12:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdba8019890 con 0x7fdbb812d080 2024-09-13T23:46:27.121 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.116+0000 7fdbb4ff9700 1 -- 172.21.15.12:0/649533180 <== mon.0 v2:172.21.15.12:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7fdba80170c0 con 0x7fdbb812d080 2024-09-13T23:46:27.126 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.124+0000 7fdbbd424700 1 -- 172.21.15.12:0/649533180 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fdb980354f0 msgr2=0x7fdb980379b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:46:27.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.124+0000 7fdbbd424700 1 --2- 172.21.15.12:0/649533180 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fdb980354f0 0x7fdb980379b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009fd0 tx=0x7fdba0008030 comp rx=0 tx=0).stop 2024-09-13T23:46:27.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.124+0000 7fdbbd424700 1 -- 172.21.15.12:0/649533180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbb812d080 msgr2=0x7fdbb811baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T23:46:27.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.124+0000 7fdbbd424700 1 --2- 172.21.15.12:0/649533180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbb812d080 0x7fdbb811baf0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdba800c480 tx=0x7fdba800c840 comp rx=0 tx=0).stop 2024-09-13T23:46:27.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.124+0000 7fdbbd424700 1 -- 172.21.15.12:0/649533180 shutdown_connections 2024-09-13T23:46:27.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.124+0000 7fdbbd424700 1 --2- 172.21.15.12:0/649533180 >> [v2:172.21.15.167:6824/1427656188,v1:172.21.15.167:6825/1427656188] conn(0x7fdb980354f0 0x7fdb980379b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:46:27.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.124+0000 7fdbbd424700 1 --2- 172.21.15.12:0/649533180 >> [v2:172.21.15.12:3300/0,v1:172.21.15.12:6789/0] conn(0x7fdbb812d080 0x7fdbb811baf0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T23:46:27.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.124+0000 7fdbbd424700 1 -- 172.21.15.12:0/649533180 >> 172.21.15.12:0/649533180 conn(0x7fdbb809be20 msgr2=0x7fdbb81313b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T23:46:27.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.124+0000 7fdbbd424700 1 -- 172.21.15.12:0/649533180 shutdown_connections 2024-09-13T23:46:27.127 INFO:teuthology.orchestra.run.smithi012.stderr:2024-09-13T23:46:27.124+0000 7fdbbd424700 1 -- 172.21.15.12:0/649533180 wait complete. 2024-09-13T23:46:27.141 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T23:46:27.141 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T23:46:27.141 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T23:46:27.142 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T23:46:27.236 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T23:46:27.236 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T23:46:27.237 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T23:46:27.327 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T23:46:27.327 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T23:46:27.327 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T23:46:30.729 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-13T23:46:30.730 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 has been restored 2024-09-13T23:46:33.429 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T23:46:33.429 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T23:46:33.430 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T23:46:33.524 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T23:46:33.524 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T23:46:33.524 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T23:46:33.524 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T23:46:33.553 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T23:46:33.553 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T23:46:33.554 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T23:46:33.554 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T23:46:33.591 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T23:46:33.592 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T23:46:33.592 DEBUG:teuthology.orchestra.run.smithi012:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | head -n 1 2024-09-13T23:46:33.611 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi012.front.sepia.ceph.com 2024-09-13T23:46:33.611 DEBUG:teuthology.orchestra.run.smithi012:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T23:46:33.815 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-13T23:46:33.815 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T23:46:34.051 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-13T23:46:34.051 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T23:46:34.282 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi167.front.sepia.ceph.com 2024-09-13T23:46:34.282 DEBUG:teuthology.orchestra.run.smithi167:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T23:46:34.473 INFO:tasks.ceph:Archiving mon data... 2024-09-13T23:46:34.474 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi/7904450/data/mon.a.tgz 2024-09-13T23:46:34.474 DEBUG:teuthology.orchestra.run.smithi012:> mktemp 2024-09-13T23:46:34.481 INFO:teuthology.orchestra.run.smithi012.stdout:/tmp/tmp.iAjs0tVRQh 2024-09-13T23:46:34.481 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.iAjs0tVRQh 2024-09-13T23:46:34.603 DEBUG:teuthology.orchestra.run.smithi012:> sudo chmod 0666 /tmp/tmp.iAjs0tVRQh 2024-09-13T23:46:34.623 DEBUG:teuthology.orchestra.remote:smithi012:/tmp/tmp.iAjs0tVRQh is 227KB 2024-09-13T23:46:34.684 DEBUG:teuthology.orchestra.run.smithi012:> rm -fr /tmp/tmp.iAjs0tVRQh 2024-09-13T23:46:34.690 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T23:46:34.691 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T23:46:34.694 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T23:46:34.710 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T23:46:34.710 DEBUG:teuthology.orchestra.run.smithi012:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T23:46:34.712 DEBUG:teuthology.orchestra.run.smithi167:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T23:46:34.732 INFO:teuthology.orchestra.run.smithi012.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T23:46:34.775 INFO:teuthology.orchestra.run.smithi167.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T23:46:34.781 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T23:46:34.783 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi/7904450/remote/smithi012/crash 2024-09-13T23:46:34.784 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T23:46:34.800 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi/7904450/remote/smithi167/crash 2024-09-13T23:46:34.801 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T23:46:34.816 INFO:tasks.ceph:Compressing logs... 2024-09-13T23:46:34.816 DEBUG:teuthology.orchestra.run.smithi012:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T23:46:34.847 DEBUG:teuthology.orchestra.run.smithi167:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T23:46:35.731 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-13T23:46:37.349 INFO:tasks.ceph:Archiving logs... 2024-09-13T23:46:37.349 DEBUG:teuthology.misc:Transferring archived files from smithi012:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi/7904450/remote/smithi012/log 2024-09-13T23:46:37.350 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T23:46:37.530 DEBUG:teuthology.misc:Transferring archived files from smithi167:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi/7904450/remote/smithi167/log 2024-09-13T23:46:37.531 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T23:46:37.928 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T23:46:37.941 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T23:46:37.941 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T23:46:37.943 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T23:46:38.025 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T23:46:38.037 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T23:46:38.037 DEBUG:teuthology.orchestra.run.smithi012:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T23:46:38.039 DEBUG:teuthology.orchestra.run.smithi167:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T23:46:38.060 INFO:teuthology.orchestra.run.smithi012.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T23:46:38.060 INFO:teuthology.orchestra.run.smithi012.stdout:============================================================================== 2024-09-13T23:46:38.060 INFO:teuthology.orchestra.run.smithi012.stdout:+hv01.front.sepi 67.205.162.81 3 u 39 64 77 0.116 0.545 0.942 2024-09-13T23:46:38.060 INFO:teuthology.orchestra.run.smithi012.stdout:+hv02.front.sepi 63.231.80.2 3 u 40 64 77 0.100 0.773 0.948 2024-09-13T23:46:38.060 INFO:teuthology.orchestra.run.smithi012.stdout:*hv03.front.sepi 74.6.168.72 3 u 43 64 77 0.141 0.541 1.126 2024-09-13T23:46:38.060 INFO:teuthology.orchestra.run.smithi012.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T23:46:38.063 INFO:teuthology.orchestra.run.smithi167.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T23:46:38.063 INFO:teuthology.orchestra.run.smithi167.stdout:============================================================================== 2024-09-13T23:46:38.063 INFO:teuthology.orchestra.run.smithi167.stdout:+hv01.front.sepi 67.205.162.81 3 u 38 64 77 0.119 0.483 2.193 2024-09-13T23:46:38.063 INFO:teuthology.orchestra.run.smithi167.stdout:*hv02.front.sepi 63.231.80.2 3 u 42 64 77 0.151 0.693 2.358 2024-09-13T23:46:38.063 INFO:teuthology.orchestra.run.smithi167.stdout:+hv03.front.sepi 74.6.168.72 3 u 39 64 77 0.099 0.478 2.373 2024-09-13T23:46:38.063 INFO:teuthology.orchestra.run.smithi167.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-13T23:46:38.064 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T23:46:38.075 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T23:46:38.075 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T23:46:38.090 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T23:46:38.104 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T23:46:38.118 INFO:teuthology.task.internal:Duration was 713.440740 seconds 2024-09-13T23:46:38.118 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T23:46:38.132 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T23:46:38.133 DEBUG:teuthology.orchestra.run.smithi012:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T23:46:38.135 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T23:46:38.185 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T23:46:38.185 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi012.front.sepia.ceph.com 2024-09-13T23:46:38.186 DEBUG:teuthology.orchestra.run.smithi012:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T23:46:38.197 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi167.front.sepia.ceph.com 2024-09-13T23:46:38.197 DEBUG:teuthology.orchestra.run.smithi167:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T23:46:38.210 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T23:46:38.210 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T23:46:38.213 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T23:46:38.251 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T23:46:38.251 DEBUG:teuthology.orchestra.run.smithi012:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T23:46:38.296 DEBUG:teuthology.orchestra.run.smithi167:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T23:46:38.368 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T23:46:38.378 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T23:46:38.378 DEBUG:teuthology.orchestra.run.smithi012:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T23:46:38.393 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T23:46:38.408 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T23:46:38.420 DEBUG:teuthology.orchestra.run.smithi012:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T23:46:38.422 DEBUG:teuthology.orchestra.run.smithi167:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T23:46:38.434 INFO:teuthology.orchestra.run.smithi012.stdout:kernel.core_pattern = core 2024-09-13T23:46:38.436 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = core 2024-09-13T23:46:38.460 DEBUG:teuthology.orchestra.run.smithi012:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T23:46:38.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T23:46:38.509 DEBUG:teuthology.orchestra.run.smithi167:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T23:46:38.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T23:46:38.515 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T23:46:38.527 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T23:46:38.528 DEBUG:teuthology.misc:Transferring archived files from smithi012:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi/7904450/remote/smithi012 2024-09-13T23:46:38.528 DEBUG:teuthology.orchestra.run.smithi012:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T23:46:38.553 DEBUG:teuthology.misc:Transferring archived files from smithi167:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-13_22:40:03-rgw-reef-distro-default-smithi/7904450/remote/smithi167 2024-09-13T23:46:38.553 DEBUG:teuthology.orchestra.run.smithi167:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T23:46:38.581 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T23:46:38.581 DEBUG:teuthology.orchestra.run.smithi012:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T23:46:38.591 DEBUG:teuthology.orchestra.run.smithi167:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T23:46:38.625 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T23:46:38.636 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T23:46:38.636 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T23:46:38.648 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T23:46:38.649 DEBUG:teuthology.orchestra.run.smithi012:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T23:46:38.651 DEBUG:teuthology.orchestra.run.smithi167:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T23:46:38.656 INFO:teuthology.orchestra.run.smithi012.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 23:46 /home/ubuntu/cephtest 2024-09-13T23:46:38.659 INFO:teuthology.orchestra.run.smithi167.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 23:46 /home/ubuntu/cephtest 2024-09-13T23:46:38.661 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T23:46:38.673 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T23:46:38.702 INFO:teuthology.run:Summary data: description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/ec-profile supported-random-distro$/{ubuntu_20.04}} duration: 713.4407398700714 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-13T23:46:38.702 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T23:46:38.794 INFO:teuthology.run:pass